[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:19 dist-test-slave-bw88 krb5kdc[18692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:19 dist-test-slave-bw88 krb5kdc[18692](info): set up 2 sockets
Aug 08 07:06:19 dist-test-slave-bw88 krb5kdc[18692](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 07:06:20.593568 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:20 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:20.718418 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43085
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:20.802769 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:20.803040 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:20.803079 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:20.804283 18707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:20.804325 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:20.804339 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:20.804375 18707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:20.804406 18707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:20.805975 18707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:20.806243 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:20.806504 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:20.812152 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:20.812136 18714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:20.812350 18707 server_base.cc:1034] running on GCE node
W20260808 07:06:20.812170 18716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:20.812750 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:20.813062 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:20.814260 18707 hybrid_clock.cc:648] HybridClock initialized: now 1786172780814249 us; error 33 us; skew 500 ppm
Aug 08 07:06:20 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:20.816744 18707 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:20.817240 18707 webserver.cc:460] Webserver started at http://127.18.63.126:42109/ using document root <none> and password file <none>
I20260808 07:06:20.817512 18707 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:20.817584 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:20.817754 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:20.818959 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "0af7e4a0c31b48db8261a050a6e697bb"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-bw88"
server_key: "151ced5ec229879cc9ee4fc188b0cd15"
server_key_iv: "b80492d73933ca648336ea1025d1c5c4"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.819357 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "0af7e4a0c31b48db8261a050a6e697bb"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-bw88"
server_key: "151ced5ec229879cc9ee4fc188b0cd15"
server_key_iv: "b80492d73933ca648336ea1025d1c5c4"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.820771 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:20.821565 18725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:20.821746 18707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:20.821861 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "0af7e4a0c31b48db8261a050a6e697bb"
format_stamp: "Formatted at 2026-08-08 07:06:20 on dist-test-slave-bw88"
server_key: "151ced5ec229879cc9ee4fc188b0cd15"
server_key_iv: "b80492d73933ca648336ea1025d1c5c4"
server_key_version: "encryptionkey@0"
I20260808 07:06:20.821936 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:20.836061 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:20.837975 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:20.838140 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:20.842723 18707 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43085
I20260808 07:06:20.842774 18801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43085 every 8 connection(s)
I20260808 07:06:20.843176 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:20.843909 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 18707
I20260808 07:06:20.844031 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:20.844358 18803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:20.844885 18685 external_mini_cluster.cc:1428] Setting key 3f36c774e803adb6e3c465eba29ae73f
I20260808 07:06:20.847827 18803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb: Bootstrap starting.
I20260808 07:06:20.848587 18803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:20.848933 18803 log.cc:824] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:20.850034 18803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb: No bootstrap required, opened a new log
I20260808 07:06:20.852459 18803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } }
I20260808 07:06:20.852667 18803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:20.852711 18803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af7e4a0c31b48db8261a050a6e697bb, State: Initialized, Role: FOLLOWER
I20260808 07:06:20.852843 18803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } }
I20260808 07:06:20.852931 18803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:20.852962 18803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:20.853010 18803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:20.853732 18803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } }
I20260808 07:06:20.853847 18803 leader_election.cc:302] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af7e4a0c31b48db8261a050a6e697bb; no voters:
I20260808 07:06:20.854019 18803 leader_election.cc:288] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:20.854064 18811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:20.854218 18811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [term 1 LEADER]: Becoming Leader. State: Replica: 0af7e4a0c31b48db8261a050a6e697bb, State: Running, Role: LEADER
I20260808 07:06:20.854321 18811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } }
I20260808 07:06:20.854324 18803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:20.854671 18811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af7e4a0c31b48db8261a050a6e697bb. Latest consensus state: current_term: 1 leader_uuid: "0af7e4a0c31b48db8261a050a6e697bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } } }
I20260808 07:06:20.854682 18812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af7e4a0c31b48db8261a050a6e697bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af7e4a0c31b48db8261a050a6e697bb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43085 } } }
I20260808 07:06:20.854770 18811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:20.854781 18812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:20.855240 18816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:20.856745 18816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:06:20 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:20.858614 18816 catalog_manager.cc:1347] Generated new cluster ID: e651ae8726864f13963eacf285339954
I20260808 07:06:20.858667 18816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:20.863370 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:20.845943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.18.63.126:43085)
0808 07:06:20.846240 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:20.846248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:20.846371 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:06:20.849235 (+ 2864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:20.849243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:20.849305 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:20.849466 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:20.851579 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.852452 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.854022 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:20.854680 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:20.854849 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:20.859362 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:20.859366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:20.859366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:20.859377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:20.861318 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.861853 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.861856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.861857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.861960 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:20.862290 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:20.862291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:20.862291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:20.862571 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:20.862733 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:20.863030 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:06:20.863167 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":81,"threads_started":1}
I20260808 07:06:20.892627 18816 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:20.893206 18816 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:20.903507 18816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af7e4a0c31b48db8261a050a6e697bb: Generated new TSK 0
I20260808 07:06:20.903757 18816 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:20.938560 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:21.046605 18840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:21.046823 18840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:21.046850 18840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:21.048130 18840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:21.048177 18840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:21.048236 18840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:21.049968 18840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:21.050246 18840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:21.050521 18840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:21.055974 18855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.056048 18850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.055945 18851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:21.056219 18840 server_base.cc:1034] running on GCE node
I20260808 07:06:21.056438 18840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:21.056761 18840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:21.057912 18840 hybrid_clock.cc:648] HybridClock initialized: now 1786172781057896 us; error 51 us; skew 500 ppm
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:21.060470 18840 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:21.060913 18840 webserver.cc:460] Webserver started at http://127.18.63.65:44539/ using document root <none> and password file <none>
I20260808 07:06:21.061180 18840 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:21.061259 18840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:21.061427 18840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:21.062486 18840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "85485ee7ccda464faff73877b79f472e"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "55f74a21e60591ae8a3058070c374ecd"
server_key_iv: "68dd54d52d95e764d2a80a815383c7bd"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.062861 18840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "85485ee7ccda464faff73877b79f472e"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "55f74a21e60591ae8a3058070c374ecd"
server_key_iv: "68dd54d52d95e764d2a80a815383c7bd"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.064217 18840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:21.064968 18863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.065137 18840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:21.065251 18840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "85485ee7ccda464faff73877b79f472e"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "55f74a21e60591ae8a3058070c374ecd"
server_key_iv: "68dd54d52d95e764d2a80a815383c7bd"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.065326 18840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:21.083541 18840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:21.086027 18840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:21.086222 18840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:21.086524 18840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:21.086943 18840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:21.087018 18840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.087064 18840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:21.087091 18840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.095590 18840 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:34859
I20260808 07:06:21.095645 19027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:34859 every 8 connection(s)
I20260808 07:06:21.095971 18840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:21.105005 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.096837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:52529 (local address 127.18.63.126:43085)
0808 07:06:21.096883 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:21.096885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:21.097120 (+ 235us) server_negotiation.cc:407] Connection header received
0808 07:06:21.097699 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:21.097700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:21.097711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:21.097765 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:21.098673 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.099168 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.100151 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.100349 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.100400 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.102562 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:21.102564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:21.102564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:21.102570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:21.103819 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.104178 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.104179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.104180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.104247 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.104471 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.104471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.104471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.104569 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:21.104620 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:21.104859 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:06:21.104937 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:21.105154 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.096801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43085 (local address 127.18.63.65:52529)
0808 07:06:21.096989 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:21.097028 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:21.097605 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:21.097819 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:21.097822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:21.098385 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:21.098588 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.098593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.099258 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.099259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.100077 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.100079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.100427 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.100428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.100557 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.100996 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:21.101004 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:21.102474 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:21.103891 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.103898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.103900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.104113 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.104308 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.104308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.104309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.104415 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.104627 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:21.104632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:21.104746 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 07:06:21.104828 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:21.104945 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260808 07:06:21.105517 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 18840
I20260808 07:06:21.105623 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:21.105696 19031 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43085
I20260808 07:06:21.105793 19031 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:21.105799 18685 external_mini_cluster.cc:1428] Setting key 7fdd600bcc2fbb84a01a722d261d64e7
I20260808 07:06:21.105991 19031 heartbeater.cc:511] Master 127.18.63.126:43085 requested a full tablet report, sending...
I20260808 07:06:21.106544 18749 ts_manager.cc:194] Registered new tserver with Master: 85485ee7ccda464faff73877b79f472e (127.18.63.65:34859)
I20260808 07:06:21.107170 18749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:52529
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:21.132714 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:21.216492 19038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:21.216701 19038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:21.216725 19038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:21.218169 19038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:21.218220 19038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:21.218248 19038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:21.219733 19038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:21.219988 19038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:21.220232 19038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:21.225642 19045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.225611 19048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.225632 19046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:21.225935 19038 server_base.cc:1034] running on GCE node
I20260808 07:06:21.226147 19038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:21.226348 19038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:21.227485 19038 hybrid_clock.cc:648] HybridClock initialized: now 1786172781227478 us; error 32 us; skew 500 ppm
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:21.230286 19038 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:21.230774 19038 webserver.cc:460] Webserver started at http://127.18.63.66:46161/ using document root <none> and password file <none>
I20260808 07:06:21.231036 19038 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:21.231108 19038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:21.231262 19038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:21.232352 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "b8279ddcdfca4dbdbdaeffa161685f29"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "66ff5696627778a7a3af0bfbbdb29508"
server_key_iv: "1bec4053fa95cdf21ca323b7fa26bb2c"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.232741 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "b8279ddcdfca4dbdbdaeffa161685f29"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "66ff5696627778a7a3af0bfbbdb29508"
server_key_iv: "1bec4053fa95cdf21ca323b7fa26bb2c"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.234324 19038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260808 07:06:21.235086 19058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.235280 19038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:21.235378 19038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "b8279ddcdfca4dbdbdaeffa161685f29"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "66ff5696627778a7a3af0bfbbdb29508"
server_key_iv: "1bec4053fa95cdf21ca323b7fa26bb2c"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.235461 19038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:21.256376 19038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:21.258831 19038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:21.259009 19038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:21.259303 19038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:21.259637 19038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:21.259697 19038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.259745 19038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:21.259769 19038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.266518 19038 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:38719
I20260808 07:06:21.266574 19209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:38719 every 8 connection(s)
I20260808 07:06:21.266958 19038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:21.268198 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19038
I20260808 07:06:21.268271 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:21.268453 18685 external_mini_cluster.cc:1428] Setting key 4cd57cbc485d528d898521d19798bf22
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:21.277530 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.267853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:51137 (local address 127.18.63.126:43085)
0808 07:06:21.267929 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:21.267931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:21.268158 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:06:21.268890 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:21.268891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:21.268904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:21.268961 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:21.270189 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.270707 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.271800 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.272018 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.272103 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.274437 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:21.274439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:21.274439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:21.274448 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:21.275957 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.276436 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.276437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.276438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.276506 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.276785 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.276786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.276787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.276917 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:21.276997 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:21.277304 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 07:06:21.277435 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:06:21.277737 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.267780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43085 (local address 127.18.63.66:51137)
0808 07:06:21.267995 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:21.268049 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:21.268773 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:21.269050 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:21.269054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:21.269851 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:21.270066 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.270072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.270816 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.270817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.271697 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.271699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.272112 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.272114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.272260 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.272821 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:21.272830 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:21.274332 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:21.276092 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.276100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.276103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.276337 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.276589 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.276590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.276590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.276707 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.277000 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:21.277008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:21.277166 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 07:06:21.277282 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:06:21.277475 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:06:21.278290 19211 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43085
I20260808 07:06:21.278390 19211 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:21.278607 19211 heartbeater.cc:511] Master 127.18.63.126:43085 requested a full tablet report, sending...
I20260808 07:06:21.279008 18749 ts_manager.cc:194] Registered new tserver with Master: b8279ddcdfca4dbdbdaeffa161685f29 (127.18.63.66:38719)
I20260808 07:06:21.279443 18749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:51137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:21.298097 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:21.385869 19220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:21.386071 19220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:21.386096 19220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:21.387243 19220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:21.387288 19220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:21.387315 19220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:21.388782 19220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:21.389038 19220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:21.389284 19220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:21.394707 19226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.394666 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:21.394685 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:21.395076 19220 server_base.cc:1034] running on GCE node
I20260808 07:06:21.395506 19220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:21.395807 19220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:21.397007 19220 hybrid_clock.cc:648] HybridClock initialized: now 1786172781396994 us; error 34 us; skew 500 ppm
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:21.400003 19220 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:21.400548 19220 webserver.cc:460] Webserver started at http://127.18.63.67:44231/ using document root <none> and password file <none>
I20260808 07:06:21.400810 19220 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:21.400888 19220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:21.401077 19220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:21.402340 19220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "1cc9b734f8314f3badbc008ea35e1c26"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "8a810f23567374a8095fb9f524c799a8"
server_key_iv: "2d62cc65273799322f593ba837d24991"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.402771 19220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "1cc9b734f8314f3badbc008ea35e1c26"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "8a810f23567374a8095fb9f524c799a8"
server_key_iv: "2d62cc65273799322f593ba837d24991"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.404527 19220 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:06:21.405452 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.405670 19220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:21.405787 19220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "1cc9b734f8314f3badbc008ea35e1c26"
format_stamp: "Formatted at 2026-08-08 07:06:21 on dist-test-slave-bw88"
server_key: "8a810f23567374a8095fb9f524c799a8"
server_key_iv: "2d62cc65273799322f593ba837d24991"
server_key_version: "encryptionkey@0"
I20260808 07:06:21.405875 19220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:21.427053 19220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:21.429615 19220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:21.429809 19220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:21.430124 19220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:21.430472 19220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:21.430531 19220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.430573 19220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:21.430601 19220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:21.437106 19220 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:43035
I20260808 07:06:21.437160 19413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:43035 every 8 connection(s)
I20260808 07:06:21.437551 19220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:21.444146 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19220
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:21.444247 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:21.444411 18685 external_mini_cluster.cc:1428] Setting key a0ab25097c595e82237593df0eedb382
I20260808 07:06:21.447642 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.438485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:34035 (local address 127.18.63.126:43085)
0808 07:06:21.438536 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:21.438538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:21.438860 (+ 322us) server_negotiation.cc:407] Connection header received
0808 07:06:21.439452 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:21.439454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:21.439468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:21.439516 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:21.440487 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.441037 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.442147 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.442371 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.442495 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.444758 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:21.444760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:21.444760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:21.444767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:21.446475 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.446855 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.446856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.446856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.446913 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.447138 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.447139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.447139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.447246 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:21.447297 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:21.447518 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 07:06:21.447579 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:06:21.447914 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.438510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43085 (local address 127.18.63.67:34035)
0808 07:06:21.438737 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:21.438773 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:21.439329 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:21.439594 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:21.439598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:21.440174 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:21.440381 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.440386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.441150 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.441167 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.442054 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:21.442057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.442510 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.442511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:21.442638 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.443181 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:21.443190 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:21.444651 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:21.446551 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.446558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.446559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.446786 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.446974 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:21.446975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:21.446975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:21.447081 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:21.447302 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:21.447307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:21.447424 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 07:06:21.447506 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:21.447642 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":139,"threads_started":1}
I20260808 07:06:21.448503 19414 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43085
I20260808 07:06:21.448613 19414 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:21.448817 19414 heartbeater.cc:511] Master 127.18.63.126:43085 requested a full tablet report, sending...
I20260808 07:06:21.449229 18749 ts_manager.cc:194] Registered new tserver with Master: 1cc9b734f8314f3badbc008ea35e1c26 (127.18.63.67:43035)
I20260808 07:06:21.449640 18749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:34035
I20260808 07:06:21.456851 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:21.482107 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.473844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.18.63.65:34859)
0808 07:06:21.474068 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:21.474074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:21.474085 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:21.474114 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:21.474115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:21.474130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:21.474205 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:21.474790 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.475615 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.476352 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.476773 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.476838 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.479054 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:21.479063 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:21.479063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:21.479070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:21.480928 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.481265 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.481268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.481268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.481329 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.481549 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.481549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.481550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.481697 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:21.481810 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:21.481873 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:06:21.481941 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":118,"threads_started":1}
W20260808 07:06:21.482609 18910 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52600
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[18692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:21.507022 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:21.499522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.18.63.65:34859)
0808 07:06:21.499593 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:21.499595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:21.499714 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:06:21.499763 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:21.499764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:21.499776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:21.499828 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:21.500376 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.500882 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.501815 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:21.502060 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:21.502127 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:21.504375 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:21.504377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:21.504377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:21.504385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:21.505984 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.506328 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.506329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.506329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.506389 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:21.506572 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:21.506573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:21.506573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:21.506672 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:21.506769 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:21.506833 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 07:06:21.506899 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:06:21.507644 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 18840
I20260808 07:06:21.519126 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19038
I20260808 07:06:21.527827 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19220
I20260808 07:06:21.537757 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 18707
2026-08-08T07:06:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2239 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[19433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[19433](info): set up 2 sockets
Aug 08 07:06:21 dist-test-slave-bw88 krb5kdc[19433](info): commencing operation
krb5kdc: starting...
W20260808 07:06:22.769269 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:22 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:22.894570 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43003
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:22.980123 19454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:22.980326 19454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:22.980348 19454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:22.981464 19454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:22.981499 19454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:22.981513 19454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:22.981526 19454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:22.981539 19454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:22.982991 19454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:22.983186 19454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:22.983366 19454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:22.988534 19464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:22.988648 19454 server_base.cc:1034] running on GCE node
W20260808 07:06:22.988624 19462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:22.988540 19459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:22.989037 19454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:22.989465 19454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:22.990634 19454 hybrid_clock.cc:648] HybridClock initialized: now 1786172782990618 us; error 29 us; skew 500 ppm
Aug 08 07:06:22 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172782, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:22.993120 19454 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:22.993605 19454 webserver.cc:460] Webserver started at http://127.18.63.126:39671/ using document root <none> and password file <none>
I20260808 07:06:22.993870 19454 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:22.993940 19454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:22.994095 19454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:22.995162 19454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "aefd4fb9cfba4452b889e851471e33d8"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-bw88"
server_key: "5d243a0273e8bf4c850201447bba613c"
server_key_iv: "f4053866d769d5e92b8778d997ff0f1e"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.995532 19454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "aefd4fb9cfba4452b889e851471e33d8"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-bw88"
server_key: "5d243a0273e8bf4c850201447bba613c"
server_key_iv: "f4053866d769d5e92b8778d997ff0f1e"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.996889 19454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:22.997685 19473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:22.997859 19454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:22.997970 19454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "aefd4fb9cfba4452b889e851471e33d8"
format_stamp: "Formatted at 2026-08-08 07:06:22 on dist-test-slave-bw88"
server_key: "5d243a0273e8bf4c850201447bba613c"
server_key_iv: "f4053866d769d5e92b8778d997ff0f1e"
server_key_version: "encryptionkey@0"
I20260808 07:06:22.998049 19454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:23.004439 19454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:23.006536 19454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:23.006713 19454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:23.010798 19454 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43003
I20260808 07:06:23.010831 19542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43003 every 8 connection(s)
I20260808 07:06:23.011175 19454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:23.012058 19543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:23.014463 19543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8: Bootstrap starting.
I20260808 07:06:23.015048 19543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:23.015349 19543 log.cc:824] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:23.016080 19543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8: No bootstrap required, opened a new log
I20260808 07:06:23.017778 19543 raft_consensus.cc:348] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } }
I20260808 07:06:23.017920 19543 raft_consensus.cc:374] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:23.017943 19543 raft_consensus.cc:729] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefd4fb9cfba4452b889e851471e33d8, State: Initialized, Role: FOLLOWER
I20260808 07:06:23.018113 19543 consensus_queue.cc:260] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } }
I20260808 07:06:23.018177 19543 raft_consensus.cc:388] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:23.018200 19543 raft_consensus.cc:482] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:23.018266 19543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:23.018790 19543 raft_consensus.cc:504] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } }
I20260808 07:06:23.018903 19543 leader_election.cc:302] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aefd4fb9cfba4452b889e851471e33d8; no voters:
I20260808 07:06:23.019080 19543 leader_election.cc:288] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:23.019147 19547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:23.019330 19547 raft_consensus.cc:686] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [term 1 LEADER]: Becoming Leader. State: Replica: aefd4fb9cfba4452b889e851471e33d8, State: Running, Role: LEADER
I20260808 07:06:23.019356 19543 sys_catalog.cc:564] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:23.019423 19547 consensus_queue.cc:237] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } }
I20260808 07:06:23.019807 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19454
I20260808 07:06:23.019912 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:23.019927 19549 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aefd4fb9cfba4452b889e851471e33d8. Latest consensus state: current_term: 1 leader_uuid: "aefd4fb9cfba4452b889e851471e33d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } } }
I20260808 07:06:23.020021 19549 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:23.020129 18685 external_mini_cluster.cc:1428] Setting key 770e102859c29566af282b6e51904b16
I20260808 07:06:23.020248 19548 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aefd4fb9cfba4452b889e851471e33d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefd4fb9cfba4452b889e851471e33d8" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43003 } } }
I20260808 07:06:23.020525 19548 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:23.020762 19559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:23.021319 19559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:23.023101 19559 catalog_manager.cc:1347] Generated new cluster ID: 29e6966cb1d3432b9f6d074d73d607b7
I20260808 07:06:23.023154 19559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:23.031293 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.020791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.18.63.126:43003)
0808 07:06:23.021065 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.021074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.021105 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:06:23.021270 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.021278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.021356 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.021608 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.022409 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.023622 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.024373 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.024807 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.024933 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.027281 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.027285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.027285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.027297 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.029713 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.030106 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.030110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.030110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.030206 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.030425 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.030426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.030426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.030672 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.030849 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.030926 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.031074 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":77,"threads_started":1}
I20260808 07:06:23.048466 19559 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:23.049013 19559 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:23.054567 19559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aefd4fb9cfba4452b889e851471e33d8: Generated new TSK 0
I20260808 07:06:23.054754 19559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:23.091825 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:23.177994 19578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:23.178192 19578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:23.178216 19578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:23.179361 19578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:23.179401 19578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:23.179427 19578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:23.180898 19578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:23.181077 19578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:23.181372 19578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:23.186642 19589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.186605 19586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.186621 19587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:23.186653 19578 server_base.cc:1034] running on GCE node
I20260808 07:06:23.187052 19578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:23.187251 19578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:23.188441 19578 hybrid_clock.cc:648] HybridClock initialized: now 1786172783188423 us; error 30 us; skew 500 ppm
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:23.191145 19578 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:23.191668 19578 webserver.cc:460] Webserver started at http://127.18.63.65:39075/ using document root <none> and password file <none>
I20260808 07:06:23.191946 19578 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:23.192041 19578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:23.192219 19578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:23.193524 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "5331f40d8881488f8982b8bc002d5b2d"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "91c10b721fad4b243b32bbbd29f1a921"
server_key_iv: "35295a66d8d08a915eabf1943ace571b"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.194000 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "5331f40d8881488f8982b8bc002d5b2d"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "91c10b721fad4b243b32bbbd29f1a921"
server_key_iv: "35295a66d8d08a915eabf1943ace571b"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.195559 19578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:23.196411 19597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.196602 19578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:23.196738 19578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "5331f40d8881488f8982b8bc002d5b2d"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "91c10b721fad4b243b32bbbd29f1a921"
server_key_iv: "35295a66d8d08a915eabf1943ace571b"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.196832 19578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:23.230212 19578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:23.232412 19578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:23.232630 19578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:23.232942 19578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:23.233332 19578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:23.233429 19578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.233489 19578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:23.233528 19578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.242077 19578 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:39065
I20260808 07:06:23.242139 19775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:39065 every 8 connection(s)
I20260808 07:06:23.242513 19578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:23.248835 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19578
I20260808 07:06:23.248919 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:23.249119 18685 external_mini_cluster.cc:1428] Setting key bbeb21583587610e1118919703db830b
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:23.252686 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.243377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:51425 (local address 127.18.63.126:43003)
0808 07:06:23.243452 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.243454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.243782 (+ 328us) server_negotiation.cc:407] Connection header received
0808 07:06:23.244795 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.244797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.244811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.244868 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.245610 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.246133 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.247174 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.247382 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.247469 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.249811 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.249813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.249814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.249821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.251241 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.251685 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.251687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.251687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.251751 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.252012 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.252013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.252014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.252136 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.252207 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.252496 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.252585 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:06:23.252877 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.243365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43003 (local address 127.18.63.65:51425)
0808 07:06:23.243586 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.243677 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.244674 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.244949 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.244953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.245230 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:23.245504 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.245509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.246234 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.246235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.247067 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.247069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.247484 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.247485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.247614 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.248154 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:23.248163 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:23.249712 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:23.251347 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.251355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.251358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.251588 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.251821 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.251822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.251823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.251937 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.252211 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:23.252219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:23.252360 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.252461 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.252621 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260808 07:06:23.253453 19776 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43003
I20260808 07:06:23.253573 19776 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:23.253782 19776 heartbeater.cc:511] Master 127.18.63.126:43003 requested a full tablet report, sending...
I20260808 07:06:23.254323 19497 ts_manager.cc:194] Registered new tserver with Master: 5331f40d8881488f8982b8bc002d5b2d (127.18.63.65:39065)
I20260808 07:06:23.254951 19497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:51425
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:23.276839 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:23.361918 19783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:23.362143 19783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:23.362170 19783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:23.363369 19783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:23.363417 19783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:23.363481 19783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:23.365180 19783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:23.365483 19783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:23.365736 19783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:23.371047 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.371109 19791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.371234 19794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:23.371675 19783 server_base.cc:1034] running on GCE node
I20260808 07:06:23.371902 19783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:23.372151 19783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:23.373292 19783 hybrid_clock.cc:648] HybridClock initialized: now 1786172783373281 us; error 26 us; skew 500 ppm
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:23.375986 19783 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:23.376418 19783 webserver.cc:460] Webserver started at http://127.18.63.66:38123/ using document root <none> and password file <none>
I20260808 07:06:23.376672 19783 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:23.376744 19783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:23.376901 19783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:23.378060 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "214a8e5dbb8247a798dc99b7ae895c99"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "ca0132dd7c1d5dc48aa27e0109b7eaec"
server_key_iv: "7a66628c6dd1bab2f49b092cd9f09e5c"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.378436 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "214a8e5dbb8247a798dc99b7ae895c99"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "ca0132dd7c1d5dc48aa27e0109b7eaec"
server_key_iv: "7a66628c6dd1bab2f49b092cd9f09e5c"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.379825 19783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:23.380616 19803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.380838 19783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:23.380949 19783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "214a8e5dbb8247a798dc99b7ae895c99"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "ca0132dd7c1d5dc48aa27e0109b7eaec"
server_key_iv: "7a66628c6dd1bab2f49b092cd9f09e5c"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.381024 19783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:23.432576 19783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:23.435261 19783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:23.435441 19783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:23.435726 19783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:23.436092 19783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:23.436151 19783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.436188 19783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:23.436237 19783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.442839 19783 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:46267
I20260808 07:06:23.443040 19956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:46267 every 8 connection(s)
I20260808 07:06:23.443288 19783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:23.453291 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19783
I20260808 07:06:23.453438 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:23.453646 18685 external_mini_cluster.cc:1428] Setting key e02b18f7563777eea088542b239dc0c6
I20260808 07:06:23.453742 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.444111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:41631 (local address 127.18.63.126:43003)
0808 07:06:23.444178 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.444181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.444379 (+ 198us) server_negotiation.cc:407] Connection header received
0808 07:06:23.445386 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.445387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.445400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.445451 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.446113 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.446631 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.448002 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.448218 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.448301 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.450663 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.450665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.450666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.450674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.452135 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.452591 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.452592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.452593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.452656 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.452923 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.452925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.452925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.453056 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.453139 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.453535 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.453638 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:06:23.454025 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.444091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43003 (local address 127.18.63.66:41631)
0808 07:06:23.444252 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.444295 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.445252 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.445524 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.445528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.445806 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:23.445999 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.446004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.446729 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.446730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.447857 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.447860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.448310 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.448311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.448446 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.449014 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:23.449023 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:23.450546 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:23.452244 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.452252 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.452254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.452498 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.452728 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.452729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.452730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.452848 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.453130 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:23.453137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:23.453285 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.453444 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.453629 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260808 07:06:23.454723 19957 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43003
I20260808 07:06:23.454834 19957 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:23.455029 19957 heartbeater.cc:511] Master 127.18.63.126:43003 requested a full tablet report, sending...
I20260808 07:06:23.455458 19497 ts_manager.cc:194] Registered new tserver with Master: 214a8e5dbb8247a798dc99b7ae895c99 (127.18.63.66:46267)
I20260808 07:06:23.455857 19497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:41631
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:23.482040 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:23.566648 19964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:23.566867 19964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:23.566893 19964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:23.568071 19964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:23.568113 19964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:23.568146 19964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:23.569766 19964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:23.570034 19964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:23.570290 19964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:23.575737 19971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.575768 19970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:23.575778 19975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:23.575928 19964 server_base.cc:1034] running on GCE node
I20260808 07:06:23.576238 19964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:23.576449 19964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:23.577581 19964 hybrid_clock.cc:648] HybridClock initialized: now 1786172783577567 us; error 30 us; skew 500 ppm
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:23.580221 19964 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:23.580724 19964 webserver.cc:460] Webserver started at http://127.18.63.67:40931/ using document root <none> and password file <none>
I20260808 07:06:23.580974 19964 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:23.581022 19964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:23.581229 19964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:23.582352 19964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "7046b1fd33a345f0a476a0827390728f"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "c20b7d99605373df1e40d524433a15a0"
server_key_iv: "203453a67bddaf71e2d9a66762c528f4"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.582737 19964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "7046b1fd33a345f0a476a0827390728f"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "c20b7d99605373df1e40d524433a15a0"
server_key_iv: "203453a67bddaf71e2d9a66762c528f4"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.584195 19964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:23.585041 19987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.585220 19964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:23.585307 19964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "7046b1fd33a345f0a476a0827390728f"
format_stamp: "Formatted at 2026-08-08 07:06:23 on dist-test-slave-bw88"
server_key: "c20b7d99605373df1e40d524433a15a0"
server_key_iv: "203453a67bddaf71e2d9a66762c528f4"
server_key_version: "encryptionkey@0"
I20260808 07:06:23.585407 19964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:23.603298 19964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:23.605721 19964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:23.605842 19964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:23.606033 19964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:23.606374 19964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:23.606410 19964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.606434 19964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:23.606477 19964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:23.612664 19964 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:36617
I20260808 07:06:23.612725 20156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:36617 every 8 connection(s)
I20260808 07:06:23.613092 19964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:23.617904 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 19964
I20260808 07:06:23.618031 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:23.618224 18685 external_mini_cluster.cc:1428] Setting key e82157b34a7959f5346aff0e69103f8a
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:23.622980 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.613907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58521 (local address 127.18.63.126:43003)
0808 07:06:23.613982 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.613984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.614191 (+ 207us) server_negotiation.cc:407] Connection header received
0808 07:06:23.615126 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.615127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.615139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.615205 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.615803 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.616298 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.617284 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.617507 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.617569 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.620192 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.620193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.620194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.620201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.621525 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.621953 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.621955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.621955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.622023 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.622300 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.622301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.622301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.622424 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.622496 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.622791 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.622877 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:06:23.623169 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.613872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43003 (local address 127.18.63.67:58521)
0808 07:06:23.614055 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.614092 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.615019 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.615262 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.615265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.615519 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:23.615723 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.615727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.616380 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.616380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.617192 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.617194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.617585 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.617586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.617703 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.618278 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:23.618309 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:23.620086 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:23.621621 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.621629 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.621631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.621861 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.622092 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:23.622093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:23.622094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:23.622219 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:23.622501 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:23.622509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:23.622649 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.622754 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.622917 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260808 07:06:23.623725 20157 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43003
I20260808 07:06:23.623836 20157 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:23.624035 20157 heartbeater.cc:511] Master 127.18.63.126:43003 requested a full tablet report, sending...
I20260808 07:06:23.624485 19497 ts_manager.cc:194] Registered new tserver with Master: 7046b1fd33a345f0a476a0827390728f (127.18.63.67:36617)
I20260808 07:06:23.624876 19497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:58521
I20260808 07:06:23.630173 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:23.638149 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.631743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.18.63.126:43003)
0808 07:06:23.631807 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.631813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.631929 (+ 116us) server_negotiation.cc:407] Connection header received
0808 07:06:23.631986 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.631986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.632001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.632057 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.632624 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.633170 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.634327 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.634511 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.634568 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.635830 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.635831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.635831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.635837 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.637133 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.637499 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.637500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.637501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.637560 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.637725 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.637725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.637725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.637860 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.637932 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.638022 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.638084 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:23.640359 19497 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:23.642113 19497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:23.649022 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.644602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:46267 (local address 127.0.0.1:41024)
0808 07:06:23.644805 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.644856 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.644933 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.645318 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.645322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.645325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:23.645413 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.645415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.646705 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.646706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.647760 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.647765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.648707 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.648785 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.648800 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.648879 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.648932 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":37,"thread_start_us":96,"threads_started":1}
I20260808 07:06:23.649226 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.645007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.18.63.67:36617)
0808 07:06:23.645217 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.645221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.645415 (+ 194us) server_negotiation.cc:407] Connection header received
0808 07:06:23.645465 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.645541 (+ 76us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.645564 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.645667 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:23.645850 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.646639 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648026 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.648718 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648779 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.648827 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.649090 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.649138 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260808 07:06:23.649290 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.644657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:36617 (local address 127.0.0.1:35292)
0808 07:06:23.645206 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.645217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.645307 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.645713 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.645714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.645715 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:23.645786 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.645788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.646759 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.646760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.647933 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.647935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.648868 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.648935 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.648939 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.648990 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.649031 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":12,"thread_start_us":88,"threads_started":1}
I20260808 07:06:23.649554 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.645008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.18.63.65:39065)
0808 07:06:23.645872 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.645877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.645886 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:23.645924 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.645926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.645940 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.646024 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:23.646169 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.647195 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648369 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.648988 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.649051 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.649106 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.649406 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.649452 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":105,"threads_started":1}
I20260808 07:06:23.649047 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.644756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.18.63.66:46267)
0808 07:06:23.645032 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.645037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.645047 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:23.645092 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.645093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.645137 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.645262 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:23.645480 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.646589 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.647855 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.648614 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.648694 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.648761 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.648905 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.648955 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":93,"threads_started":1}
I20260808 07:06:23.649718 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.644849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:39065 (local address 127.0.0.1:44472)
0808 07:06:23.645304 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.645315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.645384 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.646038 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.646039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.646041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:23.646083 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.646085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.647307 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.647308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.648281 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.648283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.649160 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.649161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.649226 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.649230 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.649278 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.649315 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":82,"threads_started":1}
I20260808 07:06:23.650302 19867 tablet_service.cc:1467] Processing CreateTablet for tablet c46f40a8aa6e4ed5a1ad8bbce356796a (DEFAULT_TABLE table=test-table [id=9453acce7cd9453f9f165cd378efaa57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:23.650317 20058 tablet_service.cc:1467] Processing CreateTablet for tablet c46f40a8aa6e4ed5a1ad8bbce356796a (DEFAULT_TABLE table=test-table [id=9453acce7cd9453f9f165cd378efaa57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:23.650406 19679 tablet_service.cc:1467] Processing CreateTablet for tablet c46f40a8aa6e4ed5a1ad8bbce356796a (DEFAULT_TABLE table=test-table [id=9453acce7cd9453f9f165cd378efaa57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:23.650596 19867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46f40a8aa6e4ed5a1ad8bbce356796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:23.650668 20058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46f40a8aa6e4ed5a1ad8bbce356796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:23.650708 19679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c46f40a8aa6e4ed5a1ad8bbce356796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:23.653030 20179 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Bootstrap starting.
I20260808 07:06:23.653647 20179 tablet_bootstrap.cc:654] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:23.653987 20180 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: Bootstrap starting.
I20260808 07:06:23.654215 20179 log.cc:824] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:23.654656 20180 tablet_bootstrap.cc:654] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:23.655200 20179 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: No bootstrap required, opened a new log
I20260808 07:06:23.655259 20180 log.cc:824] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:23.655280 20179 ts_tablet_manager.cc:1397] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:06:23.655597 20181 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Bootstrap starting.
I20260808 07:06:23.656248 20181 tablet_bootstrap.cc:654] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:23.656838 20181 log.cc:824] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:23.657445 20180 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: No bootstrap required, opened a new log
I20260808 07:06:23.657522 20180 ts_tablet_manager.cc:1397] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:06:23.657568 20179 raft_consensus.cc:348] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.657764 20179 raft_consensus.cc:374] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:23.657804 20179 raft_consensus.cc:729] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 214a8e5dbb8247a798dc99b7ae895c99, State: Initialized, Role: FOLLOWER
I20260808 07:06:23.657924 20179 consensus_queue.cc:260] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.658203 20179 ts_tablet_manager.cc:1428] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:06:23.658221 19957 heartbeater.cc:503] Master 127.18.63.126:43003 was elected leader, sending a full tablet report...
I20260808 07:06:23.659405 20180 raft_consensus.cc:348] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.659597 20180 raft_consensus.cc:374] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:23.659634 20180 raft_consensus.cc:729] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7046b1fd33a345f0a476a0827390728f, State: Initialized, Role: FOLLOWER
I20260808 07:06:23.659736 20180 consensus_queue.cc:260] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.659972 20180 ts_tablet_manager.cc:1428] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:23.660018 20157 heartbeater.cc:503] Master 127.18.63.126:43003 was elected leader, sending a full tablet report...
I20260808 07:06:23.660372 20181 tablet_bootstrap.cc:492] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: No bootstrap required, opened a new log
I20260808 07:06:23.660439 20181 ts_tablet_manager.cc:1397] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 07:06:23.662084 20181 raft_consensus.cc:348] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.662245 20181 raft_consensus.cc:374] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:23.662293 20181 raft_consensus.cc:729] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5331f40d8881488f8982b8bc002d5b2d, State: Initialized, Role: FOLLOWER
I20260808 07:06:23.662405 20181 consensus_queue.cc:260] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.662688 20181 ts_tablet_manager.cc:1428] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:06:23.662748 19776 heartbeater.cc:503] Master 127.18.63.126:43003 was elected leader, sending a full tablet report...
W20260808 07:06:23.694103 19958 tablet.cc:2369] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:23.738384 20191 raft_consensus.cc:482] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:23.738516 20191 raft_consensus.cc:504] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.738847 20191 leader_election.cc:288] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 214a8e5dbb8247a798dc99b7ae895c99 (127.18.63.66:46267), 5331f40d8881488f8982b8bc002d5b2d (127.18.63.65:39065)
I20260808 07:06:23.742031 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.739186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:39737 (local address 127.18.63.66:46267)
0808 07:06:23.739240 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.739242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.739254 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:23.739291 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.739292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.739306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.739367 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:23.739576 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.740094 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741211 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.741668 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741755 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.741765 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.741933 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.741968 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:23.742031 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.739057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:46267 (local address 127.18.63.67:39737)
0808 07:06:23.739123 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.739140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.739207 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.739382 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.739384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.739386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:23.739489 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.739492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.740180 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.740181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.741103 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.741106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.741755 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.741844 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.741847 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.741912 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.741954 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260808 07:06:23.742182 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.739205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:54081 (local address 127.18.63.65:39065)
0808 07:06:23.739269 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.739271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.739336 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:06:23.739407 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.739408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.739421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.739500 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:23.739732 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.740292 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741437 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.741826 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741909 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.741916 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.742080 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.742114 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:23.742199 20196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.739057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:39065 (local address 127.18.63.67:54081)
0808 07:06:23.739270 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:23.739280 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:23.739353 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:23.739515 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:23.739516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:23.739518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:23.739618 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.739634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.740395 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.740396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.741288 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:23.741291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.741909 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.741910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:23.742002 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.742007 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:23.742075 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:06:23.742113 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":18,"thread_start_us":123,"threads_started":1}
I20260808 07:06:23.742518 19711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46f40a8aa6e4ed5a1ad8bbce356796a" candidate_uuid: "7046b1fd33a345f0a476a0827390728f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5331f40d8881488f8982b8bc002d5b2d" is_pre_election: true
I20260808 07:06:23.742516 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46f40a8aa6e4ed5a1ad8bbce356796a" candidate_uuid: "7046b1fd33a345f0a476a0827390728f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "214a8e5dbb8247a798dc99b7ae895c99" is_pre_election: true
I20260808 07:06:23.742688 19711 raft_consensus.cc:2454] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7046b1fd33a345f0a476a0827390728f in term 0.
I20260808 07:06:23.742697 19895 raft_consensus.cc:2454] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7046b1fd33a345f0a476a0827390728f in term 0.
I20260808 07:06:23.742920 19991 leader_election.cc:302] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5331f40d8881488f8982b8bc002d5b2d, 7046b1fd33a345f0a476a0827390728f; no voters:
I20260808 07:06:23.743067 20191 raft_consensus.cc:2781] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:23.743124 20191 raft_consensus.cc:482] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:23.743147 20191 raft_consensus.cc:3037] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:06:23.743306 19777 tablet.cc:2369] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:23.744027 20191 raft_consensus.cc:504] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.744167 20191 leader_election.cc:288] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [CANDIDATE]: Term 1 election: Requested vote from peers 214a8e5dbb8247a798dc99b7ae895c99 (127.18.63.66:46267), 5331f40d8881488f8982b8bc002d5b2d (127.18.63.65:39065)
I20260808 07:06:23.744372 19711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46f40a8aa6e4ed5a1ad8bbce356796a" candidate_uuid: "7046b1fd33a345f0a476a0827390728f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5331f40d8881488f8982b8bc002d5b2d"
I20260808 07:06:23.744460 19711 raft_consensus.cc:3037] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:23.745173 19711 raft_consensus.cc:2454] T c46f40a8aa6e4ed5a1ad8bbce356796a P 5331f40d8881488f8982b8bc002d5b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7046b1fd33a345f0a476a0827390728f in term 1.
I20260808 07:06:23.745302 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c46f40a8aa6e4ed5a1ad8bbce356796a" candidate_uuid: "7046b1fd33a345f0a476a0827390728f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "214a8e5dbb8247a798dc99b7ae895c99"
I20260808 07:06:23.745422 19895 raft_consensus.cc:3037] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:23.745404 19991 leader_election.cc:302] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5331f40d8881488f8982b8bc002d5b2d, 7046b1fd33a345f0a476a0827390728f; no voters:
I20260808 07:06:23.745549 20191 raft_consensus.cc:2781] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:23.745707 20191 raft_consensus.cc:686] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [term 1 LEADER]: Becoming Leader. State: Replica: 7046b1fd33a345f0a476a0827390728f, State: Running, Role: LEADER
I20260808 07:06:23.745802 20191 consensus_queue.cc:237] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } }
I20260808 07:06:23.746378 19895 raft_consensus.cc:2454] T c46f40a8aa6e4ed5a1ad8bbce356796a P 214a8e5dbb8247a798dc99b7ae895c99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7046b1fd33a345f0a476a0827390728f in term 1.
I20260808 07:06:23.746749 19497 catalog_manager.cc:5495] T c46f40a8aa6e4ed5a1ad8bbce356796a P 7046b1fd33a345f0a476a0827390728f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7046b1fd33a345f0a476a0827390728f (127.18.63.67). New cstate: current_term: 1 leader_uuid: "7046b1fd33a345f0a476a0827390728f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "214a8e5dbb8247a798dc99b7ae895c99" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 46267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7046b1fd33a345f0a476a0827390728f" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 36617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5331f40d8881488f8982b8bc002d5b2d" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 39065 } health_report { overall_health: UNKNOWN } } }
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:23.772217 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.764834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.18.63.65:39065)
0808 07:06:23.764882 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.764884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.765008 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:06:23.765061 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.765062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.765071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.765132 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.765801 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.766288 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.766981 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.767169 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.767222 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.769607 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.769621 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.769621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.769628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.771123 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.771440 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.771443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.771443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.771534 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.771723 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.771724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.771724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.771880 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.771986 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.772081 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.772146 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:23.796860 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.789231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.18.63.65:39065)
0808 07:06:23.789301 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.789304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.789447 (+ 143us) server_negotiation.cc:407] Connection header received
0808 07:06:23.789557 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.789558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.789574 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.789638 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.790239 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.790727 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.791498 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.791698 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.791771 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.794146 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.794148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.794148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.794155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.795607 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.795981 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.795983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.795983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.796048 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.796253 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.796254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.796255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.796363 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.796465 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.796667 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.796751 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[19433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:23.821694 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:23.814234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.18.63.65:39065)
0808 07:06:23.814302 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:23.814305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:23.814420 (+ 115us) server_negotiation.cc:407] Connection header received
0808 07:06:23.814525 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:23.814528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:23.814543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:23.814605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:23.815192 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.815671 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.816414 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:23.816606 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:23.816687 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:23.818945 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:23.818947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:23.818947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:23.818955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:23.820518 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.820853 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.820854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.820855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.820917 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:23.821130 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:23.821131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:23.821131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:23.821263 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:23.821390 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:23.821509 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 07:06:23.821588 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:06:23.822292 19645 tablet_service.cc:2908] Rejecting scan request for tablet c46f40a8aa6e4ed5a1ad8bbce356796a: Uninitialized: clean time has not yet been initialized
I20260808 07:06:23.823417 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19578
I20260808 07:06:23.840245 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19783
I20260808 07:06:23.850914 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19964
I20260808 07:06:23.861060 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 19454
2026-08-08T07:06:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2322 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[20216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[20216](info): set up 2 sockets
Aug 08 07:06:23 dist-test-slave-bw88 krb5kdc[20216](info): commencing operation
krb5kdc: starting...
W20260808 07:06:25.048543 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:25.176767 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:34601
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:25.262580 20233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:25.262830 20233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:25.262866 20233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:25.263993 20233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:25.264065 20233 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:06:25.264087 20233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:25.264108 20233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:25.264163 20233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:25.264192 20233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:25.265729 20233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:34601
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:25.266027 20233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:25.266268 20233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:25.271464 20245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.271482 20241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.271435 20240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:25.271687 20233 server_base.cc:1034] running on GCE node
I20260808 07:06:25.271922 20233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:25.272218 20233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:25.273402 20233 hybrid_clock.cc:648] HybridClock initialized: now 1786172785273376 us; error 68 us; skew 500 ppm
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:25.280905 20233 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:25.281503 20233 webserver.cc:460] Webserver started at http://127.18.63.126:45623/ using document root <none> and password file <none>
I20260808 07:06:25.281762 20233 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:25.281833 20233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:25.281997 20233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:25.283085 20233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "a7c96547986a4e028cecd8359be1989e"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "c7dfcce0a2051780f35a88a3a170ddde"
server_key_iv: "dcc66c4c389606702e968d48e5bc0b87"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.283474 20233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "a7c96547986a4e028cecd8359be1989e"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "c7dfcce0a2051780f35a88a3a170ddde"
server_key_iv: "dcc66c4c389606702e968d48e5bc0b87"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.284868 20233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:25.285728 20255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.285949 20233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:25.286038 20233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "a7c96547986a4e028cecd8359be1989e"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "c7dfcce0a2051780f35a88a3a170ddde"
server_key_iv: "dcc66c4c389606702e968d48e5bc0b87"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.286125 20233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:25.301764 20233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:25.304147 20233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:25.304294 20233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:25.309870 20233 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:34601
I20260808 07:06:25.309929 20346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:34601 every 8 connection(s)
I20260808 07:06:25.310271 20233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:25.310945 20347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.313220 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 20233
I20260808 07:06:25.313359 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:25.313501 20347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Bootstrap starting.
I20260808 07:06:25.313571 18685 external_mini_cluster.cc:1428] Setting key edf5e6ca882f3daad970a2898b5af7f4
I20260808 07:06:25.314239 20347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.314566 20347 log.cc:824] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:25.315646 20347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: No bootstrap required, opened a new log
I20260808 07:06:25.318334 20347 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } }
I20260808 07:06:25.318542 20347 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.318598 20347 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c96547986a4e028cecd8359be1989e, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.318781 20347 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } }
I20260808 07:06:25.318883 20347 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:25.318922 20347 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:25.318995 20347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.319895 20347 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } }
I20260808 07:06:25.320039 20347 leader_election.cc:302] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7c96547986a4e028cecd8359be1989e; no voters:
I20260808 07:06:25.320262 20347 leader_election.cc:288] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:25.320320 20352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:25.320513 20352 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [term 1 LEADER]: Becoming Leader. State: Replica: a7c96547986a4e028cecd8359be1989e, State: Running, Role: LEADER
I20260808 07:06:25.320585 20347 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:25.320650 20352 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } }
I20260808 07:06:25.321041 20352 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c96547986a4e028cecd8359be1989e. Latest consensus state: current_term: 1 leader_uuid: "a7c96547986a4e028cecd8359be1989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } } }
I20260808 07:06:25.321208 20352 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:25.321501 20360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:25.322067 20360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:25.322808 20354 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7c96547986a4e028cecd8359be1989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7c96547986a4e028cecd8359be1989e" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34601 } } }
I20260808 07:06:25.322939 20354 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:25.324491 20360 catalog_manager.cc:1347] Generated new cluster ID: 024afbdaff2b4a81bc7c5caa3c5806ac
I20260808 07:06:25.324548 20360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:25.326133 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.314091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.18.63.126:34601)
0808 07:06:25.314291 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.314302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.314336 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:06:25.314930 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.314939 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.315013 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.315160 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.316141 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.317022 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.318243 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.318757 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.318839 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.322186 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.322191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.322191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.322204 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.324160 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.324586 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.324588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.324588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.324695 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.324905 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.324905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.324906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.325174 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.325739 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.325804 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.325927 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260808 07:06:25.333062 20360 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:25.333611 20360 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:25.342801 20360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Generated new TSK 0
I20260808 07:06:25.342975 20360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:25.376837 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34601
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:25.461542 20377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:25.461812 20377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:25.461841 20377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:25.463032 20377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:25.463074 20377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:25.463094 20377 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:25.463156 20377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:25.464586 20377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:25.464846 20377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:25.465075 20377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:25.470259 20385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.470304 20383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.470273 20382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:25.470572 20377 server_base.cc:1034] running on GCE node
I20260808 07:06:25.470793 20377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:25.470996 20377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:25.472148 20377 hybrid_clock.cc:648] HybridClock initialized: now 1786172785472127 us; error 31 us; skew 500 ppm
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:25.474967 20377 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:25.475433 20377 webserver.cc:460] Webserver started at http://127.18.63.65:41971/ using document root <none> and password file <none>
I20260808 07:06:25.475703 20377 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:25.475777 20377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:25.475944 20377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:25.477039 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "4fc285e5771eeef2e342f8d63e1f831f"
server_key_iv: "21a09e7cf7f58252ed3c179980f843ee"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.477504 20377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "4fc285e5771eeef2e342f8d63e1f831f"
server_key_iv: "21a09e7cf7f58252ed3c179980f843ee"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.478940 20377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:25.479691 20395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.479871 20377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:25.479988 20377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "4fc285e5771eeef2e342f8d63e1f831f"
server_key_iv: "21a09e7cf7f58252ed3c179980f843ee"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.480077 20377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:25.494292 20377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:25.496543 20377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:25.496734 20377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:25.497483 20377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:25.497568 20377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.497638 20377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:25.497695 20377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:25.505868 20377 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:33683
I20260808 07:06:25.505939 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:33683 every 8 connection(s)
I20260808 07:06:25.506328 20377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:25.508491 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.498185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:39989 (local address 127.18.63.126:34601)
0808 07:06:25.498273 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.498275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.498475 (+ 200us) server_negotiation.cc:407] Connection header received
0808 07:06:25.498942 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.498944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.498958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.499053 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.499885 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.500716 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.502374 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.502752 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.502837 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.505519 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.505521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.505521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.505531 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.507252 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.507624 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.507625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.507625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.507686 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.507932 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.507933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.507933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.508047 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.508109 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.508325 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.508402 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:06:25.508594 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.498116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.65:39989)
0808 07:06:25.498317 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.498357 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.498836 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.499134 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.499137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.499483 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.499763 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.499769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.500827 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.500828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.502274 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.502277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.502880 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.502881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.503035 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.503682 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.503695 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.505408 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.507331 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.507335 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.507337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.507548 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.507749 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.507749 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.507750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.507855 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.508101 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.508104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.508211 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.508290 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.508406 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260808 07:06:25.509156 20552 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34601
I20260808 07:06:25.509272 20552 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:25.509486 20552 heartbeater.cc:511] Master 127.18.63.126:34601 requested a full tablet report, sending...
I20260808 07:06:25.510089 20278 ts_manager.cc:194] Registered new tserver with Master: 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683)
I20260808 07:06:25.510957 20278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:39989
I20260808 07:06:25.512444 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 20377
I20260808 07:06:25.512566 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:25.512748 18685 external_mini_cluster.cc:1428] Setting key 65e8afcf5d34c4d8c968d2fc1435a935
I20260808 07:06:25.516625 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.510391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:58145 (local address 127.18.63.126:34601)
0808 07:06:25.510481 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.510483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.510675 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:06:25.510715 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.510716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.510725 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.510778 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.511179 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.511841 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.512539 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.512852 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.512942 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.513909 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.513911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.513911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.513918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.515509 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.515924 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.515925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.515925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.515976 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.516180 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.516181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.516181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.516274 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.516325 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.516482 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.516542 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:06:25.516732 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.510356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.65:58145)
0808 07:06:25.510584 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.510595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.510666 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.510781 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.510781 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.510998 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.511075 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.511076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.511965 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.511966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.512457 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.512458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.512940 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.512942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.513022 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.513490 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.513495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.513825 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.515601 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.515603 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.515603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.515846 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.516043 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.516044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.516044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.516116 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.516337 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.516338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.516390 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.516464 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.516574 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":152,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:25.540623 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34601
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:25.624593 20569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:25.624816 20569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:25.624858 20569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:25.626046 20569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:25.626113 20569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:25.626147 20569 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:25.626183 20569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:25.627655 20569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:25.627924 20569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:25.628163 20569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:25.633239 20578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:25.633392 20569 server_base.cc:1034] running on GCE node
W20260808 07:06:25.633265 20575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.633239 20576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:25.633766 20569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:25.633963 20569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:25.635109 20569 hybrid_clock.cc:648] HybridClock initialized: now 1786172785635099 us; error 29 us; skew 500 ppm
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:25.637656 20569 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:25.638134 20569 webserver.cc:460] Webserver started at http://127.18.63.66:45259/ using document root <none> and password file <none>
I20260808 07:06:25.638391 20569 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:25.638463 20569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:25.638616 20569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:25.639735 20569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "59790674c8cc4c1e87bed699ed6d7eb1"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "3b4a0183d2697b23a0a8d647a090c1d3"
server_key_iv: "2cb74763870d4d91db31a97f95afd1b6"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.640096 20569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "59790674c8cc4c1e87bed699ed6d7eb1"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "3b4a0183d2697b23a0a8d647a090c1d3"
server_key_iv: "2cb74763870d4d91db31a97f95afd1b6"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.641474 20569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:25.642210 20592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.642386 20569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.642493 20569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "59790674c8cc4c1e87bed699ed6d7eb1"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "3b4a0183d2697b23a0a8d647a090c1d3"
server_key_iv: "2cb74763870d4d91db31a97f95afd1b6"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.642570 20569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:25.670220 20569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:25.672101 20569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:25.672273 20569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:25.672869 20569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:25.672930 20569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.672966 20569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:25.673049 20569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:25.680899 20569 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:33873
I20260808 07:06:25.680979 20751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:33873 every 8 connection(s)
I20260808 07:06:25.681423 20569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:25.684144 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.673388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.66:33213)
0808 07:06:25.673595 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.673644 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.674123 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.674398 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.674402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.674641 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.674874 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.674880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.675671 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.675673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.676547 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.676549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.677002 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.677003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.677185 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.677689 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.677701 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.680070 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.682038 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.682045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.682047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.682333 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.682558 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.682559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.682559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.682742 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.683345 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.683382 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.683602 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.683716 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.683911 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":131,"threads_started":1}
I20260808 07:06:25.684481 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.673404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:33213 (local address 127.18.63.126:34601)
0808 07:06:25.673485 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.673488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.673744 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:06:25.674228 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.674230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.674244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.674327 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.674972 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.675546 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.676663 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.676901 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.676978 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.680189 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.680191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.680191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.680200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.681922 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.682417 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.682419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.682419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.682476 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.683094 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.683095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.683095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.683279 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.683355 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.684297 (+ 942us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.684382 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:06:25.685168 20752 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34601
I20260808 07:06:25.685261 20752 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:25.685477 20752 heartbeater.cc:511] Master 127.18.63.126:34601 requested a full tablet report, sending...
I20260808 07:06:25.685475 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 20569
I20260808 07:06:25.685588 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:25.685789 18685 external_mini_cluster.cc:1428] Setting key 11602ba9f84351098a82fc6d8abaebf9
I20260808 07:06:25.685907 20279 ts_manager.cc:194] Registered new tserver with Master: 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873)
I20260808 07:06:25.686290 20279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:33213
I20260808 07:06:25.694427 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.686265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:36061 (local address 127.18.63.126:34601)
0808 07:06:25.686299 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.686301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.688414 (+ 2113us) server_negotiation.cc:407] Connection header received
0808 07:06:25.688492 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.688492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.688511 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.688577 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.689005 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.689577 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.690361 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.690569 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.690642 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.691655 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.691656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.691656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.691664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.693041 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.693568 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.693570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.693570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.693626 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.693836 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.693837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.693838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.693985 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.694079 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.694257 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.694334 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:06:25.694439 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.686331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.66:36061)
0808 07:06:25.686470 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.686481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.688436 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.688586 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.688587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.688843 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.688925 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.688927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.689756 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.689757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.690277 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.690278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.690643 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.690644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.690723 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.691239 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.691244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.691560 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.693241 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.693242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.693243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.693480 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.693704 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.693705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.693705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.693771 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.694079 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.694080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.694140 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.694229 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.694335 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:25.714172 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34601
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:25.797068 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:25.797286 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:25.797309 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:25.798506 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:25.798547 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:25.798568 20771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:25.798586 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:25.800107 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:25.800343 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:25.800590 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:25.805747 20778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.805840 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:25.805747 20781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:25.805967 20771 server_base.cc:1034] running on GCE node
I20260808 07:06:25.806196 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:25.806401 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:25.807562 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786172785807549 us; error 40 us; skew 500 ppm
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:25.810267 20771 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:25.810791 20771 webserver.cc:460] Webserver started at http://127.18.63.67:43085/ using document root <none> and password file <none>
I20260808 07:06:25.811034 20771 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:25.811105 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:25.811254 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:25.812319 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "d64a89dff0e345acb9caf66fad273f2c"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "a2bb0631c8b43a72cf35b7c483503e76"
server_key_iv: "7add2c2de120a9a911732634018fc27f"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.812711 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "d64a89dff0e345acb9caf66fad273f2c"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "a2bb0631c8b43a72cf35b7c483503e76"
server_key_iv: "7add2c2de120a9a911732634018fc27f"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.814211 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:25.814939 20789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.815130 20771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:25.815251 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "d64a89dff0e345acb9caf66fad273f2c"
format_stamp: "Formatted at 2026-08-08 07:06:25 on dist-test-slave-bw88"
server_key: "a2bb0631c8b43a72cf35b7c483503e76"
server_key_iv: "7add2c2de120a9a911732634018fc27f"
server_key_version: "encryptionkey@0"
I20260808 07:06:25.815330 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:25.836493 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:25.838537 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:25.838709 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:25.839324 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:25.839388 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.839432 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:25.839461 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:25.847332 20771 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46373
I20260808 07:06:25.847414 20949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46373 every 8 connection(s)
I20260808 07:06:25.847806 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:25.849468 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 20771
I20260808 07:06:25.849586 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:25.849774 18685 external_mini_cluster.cc:1428] Setting key 88912c1be29e1058e51f9deea97a145c
I20260808 07:06:25.850452 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.840088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:53073 (local address 127.18.63.126:34601)
0808 07:06:25.840166 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.840169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.840181 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:25.840612 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.840613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.840627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.840678 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.841567 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.842073 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.843470 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.843747 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.843832 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.847342 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.847344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.847345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.847352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.848868 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.849311 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.849312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.849313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.849390 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.849684 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.849685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.849685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.849807 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.849887 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.850256 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.850350 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:06:25.850505 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.839730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.67:53073)
0808 07:06:25.839900 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.839942 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.840499 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.840754 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.840757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.841131 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.841470 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.841477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.842182 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.842183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.843377 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.843380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.843845 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.843846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.843983 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.844484 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.844497 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.847245 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.848965 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.848973 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.848975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.849198 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.849467 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.849468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.849468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.849616 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.849878 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.849885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.850025 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.850161 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.850315 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260808 07:06:25.850962 20950 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34601
I20260808 07:06:25.851063 20950 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:25.851260 20950 heartbeater.cc:511] Master 127.18.63.126:34601 requested a full tablet report, sending...
I20260808 07:06:25.851629 20278 ts_manager.cc:194] Registered new tserver with Master: d64a89dff0e345acb9caf66fad273f2c (127.18.63.67:46373)
I20260808 07:06:25.852154 20278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:53073
I20260808 07:06:25.857198 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.852023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:47475 (local address 127.18.63.126:34601)
0808 07:06:25.852058 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.852060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.852206 (+ 146us) server_negotiation.cc:407] Connection header received
0808 07:06:25.852233 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.852234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.852245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.852304 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.852704 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.853264 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.854018 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.854196 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.854247 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.855050 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.855051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.855051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.855057 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.856382 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.856674 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.856674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.856675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.856715 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.856863 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.856863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.856864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.856941 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.856983 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.857090 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.857142 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:06:25.857216 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.851974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.18.63.67:47475)
0808 07:06:25.852128 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.852137 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.852192 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.852348 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.852350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.852571 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:25.852623 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.852625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.853409 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.853410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.853946 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.853948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.854261 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.854262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.854319 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.854712 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:25.854716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:25.854974 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:25.856457 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.856458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.856458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.856617 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.856768 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:25.856769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:25.856769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:25.856820 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:25.856987 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:25.856988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:25.857028 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.857076 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.857149 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260808 07:06:25.861423 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:06:25 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:25.886946 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.878806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.18.63.126:34601)
0808 07:06:25.878877 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.878879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.878890 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:25.878914 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.878914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.878923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.878976 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:25.879804 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.880319 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.881201 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.881660 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.881718 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.884171 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:25.884173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:25.884173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:25.884181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:25.885886 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.886269 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.886270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.886270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.886330 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:25.886512 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:25.886513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:25.886513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:25.886643 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:25.886727 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.886812 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.886875 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:25.888713 20278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:25.890614 20278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:25.897038 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.892962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:33873 (local address 127.0.0.1:43360)
0808 07:06:25.893084 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.893129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.893217 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.893929 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.893933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.893935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:25.894005 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.894007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.895015 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.895016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.895990 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.895992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.896699 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.896700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.896782 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.896794 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.896893 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.896953 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260808 07:06:25.897048 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.893123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.18.63.66:33873)
0808 07:06:25.893656 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.893660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.893673 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:25.893722 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.893725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.893745 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.893843 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:25.894092 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.894929 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.896086 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.896595 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.896686 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.896749 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.896912 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.896952 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":98,"threads_started":1}
I20260808 07:06:25.897921 20656 tablet_service.cc:1467] Processing CreateTablet for tablet 434b16d97b4943089b73ced9ec11db25 (DEFAULT_TABLE table=test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.898363 20656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434b16d97b4943089b73ced9ec11db25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.898458 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.893216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33683 (local address 127.0.0.1:37804)
0808 07:06:25.893622 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.893633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.893737 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.894014 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.894014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.894016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:25.894074 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.894076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.895559 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.895561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.897020 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.897023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.898200 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.898201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.898271 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.898275 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.898331 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.898375 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":35,"thread_start_us":77,"threads_started":1}
I20260808 07:06:25.898516 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.893419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.18.63.65:33683)
0808 07:06:25.893616 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.893622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.893705 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:06:25.893837 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.893839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.893857 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.893922 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:25.894165 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.895442 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.897194 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.898014 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.898068 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.898128 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.898358 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.898393 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260808 07:06:25.899209 20463 tablet_service.cc:1467] Processing CreateTablet for tablet 434b16d97b4943089b73ced9ec11db25 (DEFAULT_TABLE table=test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.899595 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434b16d97b4943089b73ced9ec11db25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.902269 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.893417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46373 (local address 127.0.0.1:52646)
0808 07:06:25.893680 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.893692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.893750 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.894364 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.894365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.894368 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:25.894411 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.894412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.895750 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.895751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.897027 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.897028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.902024 (+ 4996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.902025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.902071 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.902074 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.902156 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.902199 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":33,"thread_start_us":134,"threads_started":1}
I20260808 07:06:25.902415 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.893545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.18.63.67:46373)
0808 07:06:25.894176 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.894181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.894191 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:25.894235 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.894237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.894254 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.894351 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:25.894507 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.895655 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.901420 (+ 5765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.901900 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.901940 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.901972 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.902174 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.902220 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":109,"threads_started":1}
I20260808 07:06:25.903051 20866 tablet_service.cc:1467] Processing CreateTablet for tablet 434b16d97b4943089b73ced9ec11db25 (DEFAULT_TABLE table=test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.903390 20866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 434b16d97b4943089b73ced9ec11db25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.903739 20992 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Bootstrap starting.
I20260808 07:06:25.904417 20992 tablet_bootstrap.cc:654] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.904816 20992 log.cc:824] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:25.905787 20992 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: No bootstrap required, opened a new log
I20260808 07:06:25.905849 20992 ts_tablet_manager.cc:1397] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:25.906243 20990 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Bootstrap starting.
I20260808 07:06:25.906261 20995 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Bootstrap starting.
I20260808 07:06:25.906807 20990 tablet_bootstrap.cc:654] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.907109 20990 log.cc:824] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:25.907205 20995 tablet_bootstrap.cc:654] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.907583 20995 log.cc:824] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:25.908071 20990 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: No bootstrap required, opened a new log
I20260808 07:06:25.908205 20990 ts_tablet_manager.cc:1397] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 07:06:25.908200 20992 raft_consensus.cc:348] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.908423 20992 raft_consensus.cc:374] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.908476 20992 raft_consensus.cc:729] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40d9850f0edd4491b2a4ebaea5cfb7cd, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.908599 20995 tablet_bootstrap.cc:492] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: No bootstrap required, opened a new log
I20260808 07:06:25.908696 20992 consensus_queue.cc:260] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.908810 20995 ts_tablet_manager.cc:1397] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:25.909015 20992 ts_tablet_manager.cc:1428] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:25.909166 20552 heartbeater.cc:503] Master 127.18.63.126:34601 was elected leader, sending a full tablet report...
I20260808 07:06:25.910442 20990 raft_consensus.cc:348] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.910599 20990 raft_consensus.cc:374] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.910641 20990 raft_consensus.cc:729] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59790674c8cc4c1e87bed699ed6d7eb1, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.910653 20995 raft_consensus.cc:348] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.910756 20990 consensus_queue.cc:260] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.910835 20995 raft_consensus.cc:374] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.910873 20995 raft_consensus.cc:729] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64a89dff0e345acb9caf66fad273f2c, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.911027 20990 ts_tablet_manager.cc:1428] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:25.910981 20995 consensus_queue.cc:260] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.911120 20752 heartbeater.cc:503] Master 127.18.63.126:34601 was elected leader, sending a full tablet report...
I20260808 07:06:25.911307 20995 ts_tablet_manager.cc:1428] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:06:25.911423 20950 heartbeater.cc:503] Master 127.18.63.126:34601 was elected leader, sending a full tablet report...
I20260808 07:06:25.925398 21003 raft_consensus.cc:482] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:25.925573 21003 raft_consensus.cc:504] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.925908 21003 leader_election.cc:288] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873), 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683)
I20260808 07:06:25.929207 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.926205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33683 (local address 127.18.63.67:46335)
0808 07:06:25.926389 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.926399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.926463 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.926602 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.926603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.926605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:25.926677 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.926679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.927513 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.927514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.928311 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.928314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.929011 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.929012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.929075 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.929078 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.929116 (+ 38us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.929148 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
I20260808 07:06:25.929282 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.926329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:46335 (local address 127.18.63.65:33683)
0808 07:06:25.926374 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.926376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.926471 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:06:25.926503 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.926504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.926513 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.926574 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:25.926775 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.927410 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.928424 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.928902 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.928945 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.928952 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.929148 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.929197 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:06:25.929415 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.926074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:33873 (local address 127.18.63.67:47835)
0808 07:06:25.926140 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:25.926155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:25.926221 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:25.926438 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:25.926438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:25.926440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:25.926526 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.926528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.927258 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.927259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.928224 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:25.928227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.929023 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.929024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:25.929179 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.929182 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:25.929249 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:06:25.929291 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":4}
I20260808 07:06:25.929437 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.926165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:47835 (local address 127.18.63.66:33873)
0808 07:06:25.926227 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.926229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.926243 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:06:25.926324 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.926325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.926335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.926385 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:25.926611 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.927149 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.928334 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.928902 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.929018 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.929027 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.929281 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.929331 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:06:25.929580 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434b16d97b4943089b73ced9ec11db25" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" is_pre_election: true
I20260808 07:06:25.929768 20491 raft_consensus.cc:2454] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 0.
I20260808 07:06:25.929761 20685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434b16d97b4943089b73ced9ec11db25" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" is_pre_election: true
I20260808 07:06:25.929888 20685 raft_consensus.cc:2454] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 0.
I20260808 07:06:25.930037 20791 leader_election.cc:302] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d9850f0edd4491b2a4ebaea5cfb7cd, d64a89dff0e345acb9caf66fad273f2c; no voters:
I20260808 07:06:25.930197 21003 raft_consensus.cc:2781] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:25.930275 21003 raft_consensus.cc:482] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:25.930295 21003 raft_consensus.cc:3037] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.931054 21003 raft_consensus.cc:504] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.931206 21003 leader_election.cc:288] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 election: Requested vote from peers 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873), 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683)
I20260808 07:06:25.931386 20685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434b16d97b4943089b73ced9ec11db25" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59790674c8cc4c1e87bed699ed6d7eb1"
I20260808 07:06:25.931448 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "434b16d97b4943089b73ced9ec11db25" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
I20260808 07:06:25.931520 20685 raft_consensus.cc:3037] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.931546 20491 raft_consensus.cc:3037] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:06:25.932067 20753 tablet.cc:2369] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:25.932384 20491 raft_consensus.cc:2454] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 1.
I20260808 07:06:25.932447 20685 raft_consensus.cc:2454] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 1.
I20260808 07:06:25.932626 20791 leader_election.cc:302] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59790674c8cc4c1e87bed699ed6d7eb1, d64a89dff0e345acb9caf66fad273f2c; no voters:
I20260808 07:06:25.932739 21003 raft_consensus.cc:2781] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:25.932902 21003 raft_consensus.cc:686] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 1 LEADER]: Becoming Leader. State: Replica: d64a89dff0e345acb9caf66fad273f2c, State: Running, Role: LEADER
I20260808 07:06:25.932976 21003 consensus_queue.cc:237] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.933770 20279 catalog_manager.cc:5495] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to d64a89dff0e345acb9caf66fad273f2c (127.18.63.67). New cstate: current_term: 1 leader_uuid: "d64a89dff0e345acb9caf66fad273f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:25.944268 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:25.941778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.18.63.67:46373)
0808 07:06:25.941838 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:25.941840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:25.941848 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:25.941875 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:25.941876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:25.941894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:25.941970 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:25.942124 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.942572 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.943369 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:25.943573 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:25.943685 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:25.943906 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:25.944006 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:25.944095 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:25.944160 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:06:25.944180 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:06:25.946564 20491 raft_consensus.cc:1261] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Refusing update from remote peer d64a89dff0e345acb9caf66fad273f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:25.946564 20685 raft_consensus.cc:1261] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Refusing update from remote peer d64a89dff0e345acb9caf66fad273f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:25.946902 21003 consensus_queue.cc:1035] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:25.947055 21005 consensus_queue.cc:1035] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:25.949434 21009 mvcc.cc:205] Tried to move back new op lower bound from 7316163731234316288 to 7316163731182198784. Current Snapshot: MvccSnapshot[applied={T|T < 7316163731234316288}]
I20260808 07:06:25.951118 21010 mvcc.cc:205] Tried to move back new op lower bound from 7316163731234316288 to 7316163731182198784. Current Snapshot: MvccSnapshot[applied={T|T < 7316163731234316288 or (T in {7316163731234316288})}]
I20260808 07:06:25.953315 20278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:25.953460 20278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:25.954267 20278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Sending DeleteTablet for 3 replicas of tablet 434b16d97b4943089b73ced9ec11db25
I20260808 07:06:25.954701 20656 tablet_service.cc:1514] Processing DeleteTablet for tablet 434b16d97b4943089b73ced9ec11db25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:25 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:43360
I20260808 07:06:25.954713 20866 tablet_service.cc:1514] Processing DeleteTablet for tablet 434b16d97b4943089b73ced9ec11db25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:25 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:52646
I20260808 07:06:25.954865 20278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:25.955005 20278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:25.954999 21023 tablet_replica.cc:330] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: stopping tablet replica
I20260808 07:06:25.955029 20463 tablet_service.cc:1514] Processing DeleteTablet for tablet 434b16d97b4943089b73ced9ec11db25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:25 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37804
I20260808 07:06:25.955104 21023 raft_consensus.cc:2229] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:25.955200 21023 raft_consensus.cc:2258] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:25.955248 21026 tablet_replica.cc:330] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: stopping tablet replica
I20260808 07:06:25.955345 21026 raft_consensus.cc:2229] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:25.955451 21026 raft_consensus.cc:2258] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:25.955678 21023 ts_tablet_manager.cc:1905] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:25.956252 21026 ts_tablet_manager.cc:1905] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:25.956552 20866 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6108675f2947b786e39efec76cf991 (DEFAULT_TABLE table=test-table [id=5bad40b105f74038a165d86e095aa6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.956666 20866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6108675f2947b786e39efec76cf991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.956687 20656 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6108675f2947b786e39efec76cf991 (DEFAULT_TABLE table=test-table [id=5bad40b105f74038a165d86e095aa6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.956785 20656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6108675f2947b786e39efec76cf991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.957734 20990 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Bootstrap starting.
I20260808 07:06:25.958056 20990 tablet_bootstrap.cc:654] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.958806 20990 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: No bootstrap required, opened a new log
I20260808 07:06:25.958874 20990 ts_tablet_manager.cc:1397] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:25.959038 20990 raft_consensus.cc:348] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.959112 20990 raft_consensus.cc:374] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.959136 20990 raft_consensus.cc:729] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59790674c8cc4c1e87bed699ed6d7eb1, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.959198 21026 ts_tablet_manager.cc:1918] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:25.959208 20990 consensus_queue.cc:260] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.959247 20463 tablet_service.cc:1467] Processing CreateTablet for tablet 2e6108675f2947b786e39efec76cf991 (DEFAULT_TABLE table=test-table [id=5bad40b105f74038a165d86e095aa6a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:25.959281 21026 log.cc:1196] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/434b16d97b4943089b73ced9ec11db25
I20260808 07:06:25.959298 20990 ts_tablet_manager.cc:1428] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:25.959350 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e6108675f2947b786e39efec76cf991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:25.959564 21026 ts_tablet_manager.cc:1939] T 434b16d97b4943089b73ced9ec11db25 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting consensus metadata
I20260808 07:06:25.959959 20257 catalog_manager.cc:4841] TS 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683): tablet 434b16d97b4943089b73ced9ec11db25 (table test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]) successfully deleted
I20260808 07:06:25.960038 21023 ts_tablet_manager.cc:1918] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:25.960122 21023 log.cc:1196] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/434b16d97b4943089b73ced9ec11db25
I20260808 07:06:25.960376 21023 ts_tablet_manager.cc:1939] T 434b16d97b4943089b73ced9ec11db25 P d64a89dff0e345acb9caf66fad273f2c: Deleting consensus metadata
I20260808 07:06:25.960740 20992 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Bootstrap starting.
I20260808 07:06:25.960774 20257 catalog_manager.cc:4841] TS d64a89dff0e345acb9caf66fad273f2c (127.18.63.67:46373): tablet 434b16d97b4943089b73ced9ec11db25 (table test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]) successfully deleted
I20260808 07:06:25.960796 20995 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Bootstrap starting.
I20260808 07:06:25.961130 20992 tablet_bootstrap.cc:654] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.961148 20995 tablet_bootstrap.cc:654] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:25.961609 21021 tablet_replica.cc:330] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: stopping tablet replica
I20260808 07:06:25.961725 21021 raft_consensus.cc:2229] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:25.961923 20995 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: No bootstrap required, opened a new log
I20260808 07:06:25.961977 20995 ts_tablet_manager.cc:1397] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:25.961983 20992 tablet_bootstrap.cc:492] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: No bootstrap required, opened a new log
I20260808 07:06:25.962036 20992 ts_tablet_manager.cc:1397] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:25.962165 20995 raft_consensus.cc:348] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.962198 20992 raft_consensus.cc:348] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.962247 20995 raft_consensus.cc:374] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.962257 20992 raft_consensus.cc:374] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:25.962275 20995 raft_consensus.cc:729] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64a89dff0e345acb9caf66fad273f2c, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.962293 20992 raft_consensus.cc:729] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40d9850f0edd4491b2a4ebaea5cfb7cd, State: Initialized, Role: FOLLOWER
I20260808 07:06:25.962335 20995 consensus_queue.cc:260] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.962347 20992 consensus_queue.cc:260] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.962427 20995 ts_tablet_manager.cc:1428] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:25.962427 20992 ts_tablet_manager.cc:1428] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:25.962880 21021 raft_consensus.cc:2258] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:25.963277 21021 ts_tablet_manager.cc:1905] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:25.964468 21021 ts_tablet_manager.cc:1918] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:25.964529 21021 log.cc:1196] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/434b16d97b4943089b73ced9ec11db25
I20260808 07:06:25.964859 21021 ts_tablet_manager.cc:1939] T 434b16d97b4943089b73ced9ec11db25 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting consensus metadata
I20260808 07:06:25.965256 20257 catalog_manager.cc:4841] TS 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873): tablet 434b16d97b4943089b73ced9ec11db25 (table test-table [id=a29b82df6cf94cd09e4ebcca653ab3dd]) successfully deleted
I20260808 07:06:25.989583 21003 raft_consensus.cc:482] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:25.989707 21003 raft_consensus.cc:504] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.989878 21003 leader_election.cc:288] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683), 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873)
I20260808 07:06:25.990082 20685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6108675f2947b786e39efec76cf991" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" is_pre_election: true
I20260808 07:06:25.990098 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6108675f2947b786e39efec76cf991" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" is_pre_election: true
I20260808 07:06:25.990203 20491 raft_consensus.cc:2454] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 0.
I20260808 07:06:25.990200 20685 raft_consensus.cc:2454] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 0.
I20260808 07:06:25.990370 20791 leader_election.cc:302] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d9850f0edd4491b2a4ebaea5cfb7cd, d64a89dff0e345acb9caf66fad273f2c; no voters:
I20260808 07:06:25.990497 21003 raft_consensus.cc:2781] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:25.990541 21003 raft_consensus.cc:482] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:25.990569 21003 raft_consensus.cc:3037] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.991261 21003 raft_consensus.cc:504] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.991394 21003 leader_election.cc:288] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 election: Requested vote from peers 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683), 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873)
I20260808 07:06:25.991580 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6108675f2947b786e39efec76cf991" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
I20260808 07:06:25.991600 20685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e6108675f2947b786e39efec76cf991" candidate_uuid: "d64a89dff0e345acb9caf66fad273f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59790674c8cc4c1e87bed699ed6d7eb1"
I20260808 07:06:25.991683 20685 raft_consensus.cc:3037] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.991679 20491 raft_consensus.cc:3037] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:25.992337 20491 raft_consensus.cc:2454] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 1.
I20260808 07:06:25.992494 20685 raft_consensus.cc:2454] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64a89dff0e345acb9caf66fad273f2c in term 1.
I20260808 07:06:25.992525 20791 leader_election.cc:302] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d9850f0edd4491b2a4ebaea5cfb7cd, d64a89dff0e345acb9caf66fad273f2c; no voters:
I20260808 07:06:25.992673 21003 raft_consensus.cc:2781] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:25.992734 21003 raft_consensus.cc:686] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 1 LEADER]: Becoming Leader. State: Replica: d64a89dff0e345acb9caf66fad273f2c, State: Running, Role: LEADER
I20260808 07:06:25.992790 21003 consensus_queue.cc:237] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } }
I20260808 07:06:25.993371 20278 catalog_manager.cc:5495] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to d64a89dff0e345acb9caf66fad273f2c (127.18.63.67). New cstate: current_term: 1 leader_uuid: "d64a89dff0e345acb9caf66fad273f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } health_report { overall_health: HEALTHY } } }
W20260808 07:06:26.007265 20554 tablet.cc:2369] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:06:26 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:26.026880 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.018254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.18.63.126:34601)
0808 07:06:26.018321 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.018324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.018479 (+ 155us) server_negotiation.cc:407] Connection header received
0808 07:06:26.018561 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.018562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.018572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.018651 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:26.019346 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.019849 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.021016 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.021230 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.021286 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.023940 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:26.023941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:26.023942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:26.023948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:26.025524 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.026028 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.026029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.026030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.026122 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.026387 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.026388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.026388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.026493 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:26.026548 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.026740 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.026813 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:06:26.027027 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.018186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34601 (local address 127.0.0.1:41122)
0808 07:06:26.018375 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.018391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.018492 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.018684 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.018685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.019164 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:26.019249 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.019253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.019941 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.019942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.020913 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.020917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.021318 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.021319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.021408 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.022048 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:26.022054 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:26.023839 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:26.025605 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:26.025613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:26.025615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:26.025899 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:26.026229 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:26.026230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:26.026230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:26.026314 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:26.026564 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:26.026571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:26.026629 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.026716 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.026894 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":54,"thread_start_us":113,"threads_started":1}
I20260808 07:06:26.030458 20278 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:41122:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:06:26.030781 20278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:26.032516 20866 tablet_service.cc:1467] Processing CreateTablet for tablet 6109dea8e8cf4e29a7c370171b126739 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e63ad5427824ebc835330e7a70d19de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:26.032703 20866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6109dea8e8cf4e29a7c370171b126739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:26.032684 20656 tablet_service.cc:1467] Processing CreateTablet for tablet 6109dea8e8cf4e29a7c370171b126739 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e63ad5427824ebc835330e7a70d19de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:26.032685 20463 tablet_service.cc:1467] Processing CreateTablet for tablet 6109dea8e8cf4e29a7c370171b126739 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4e63ad5427824ebc835330e7a70d19de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:26.032825 20656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6109dea8e8cf4e29a7c370171b126739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:26.032861 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6109dea8e8cf4e29a7c370171b126739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:26.034493 20995 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: Bootstrap starting.
I20260808 07:06:26.034686 20992 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Bootstrap starting.
I20260808 07:06:26.034926 20995 tablet_bootstrap.cc:654] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:26.035166 20992 tablet_bootstrap.cc:654] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:26.035894 20995 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: No bootstrap required, opened a new log
I20260808 07:06:26.035933 20995 ts_tablet_manager.cc:1397] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:26.036079 20995 raft_consensus.cc:348] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.036132 20995 raft_consensus.cc:374] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:26.036146 20995 raft_consensus.cc:729] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64a89dff0e345acb9caf66fad273f2c, State: Initialized, Role: FOLLOWER
I20260808 07:06:26.036181 20995 consensus_queue.cc:260] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.036231 20992 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: No bootstrap required, opened a new log
I20260808 07:06:26.036289 20995 ts_tablet_manager.cc:1428] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:26.036288 20992 ts_tablet_manager.cc:1397] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:26.036374 21003 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.036417 21003 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.036474 20992 raft_consensus.cc:348] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.036543 20992 raft_consensus.cc:374] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:26.036571 20992 raft_consensus.cc:729] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40d9850f0edd4491b2a4ebaea5cfb7cd, State: Initialized, Role: FOLLOWER
I20260808 07:06:26.036639 20992 consensus_queue.cc:260] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.036736 20992 ts_tablet_manager.cc:1428] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:26.036861 20990 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: Bootstrap starting.
I20260808 07:06:26.036953 20999 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.037076 20999 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.037397 20990 tablet_bootstrap.cc:654] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:26.038389 20990 tablet_bootstrap.cc:492] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: No bootstrap required, opened a new log
I20260808 07:06:26.038455 20990 ts_tablet_manager.cc:1397] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:26.038640 20990 raft_consensus.cc:348] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.038725 20990 raft_consensus.cc:374] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:26.038750 20990 raft_consensus.cc:729] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59790674c8cc4c1e87bed699ed6d7eb1, State: Initialized, Role: FOLLOWER
I20260808 07:06:26.038798 20990 consensus_queue.cc:260] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.038904 20990 ts_tablet_manager.cc:1428] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:26.039075 21002 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.039177 21002 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 07:06:26.098416 20951 tablet.cc:2369] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:26.175772 21002 raft_consensus.cc:482] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:26.175921 21002 raft_consensus.cc:504] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.176359 21002 leader_election.cc:288] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683), d64a89dff0e345acb9caf66fad273f2c (127.18.63.67:46373)
I20260808 07:06:26.180260 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.176759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:57881 (local address 127.18.63.65:33683)
0808 07:06:26.176818 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.176820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.176857 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:06:26.176896 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.176897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.176908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.176967 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:26.177156 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.177948 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.179485 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.179913 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.179993 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.180002 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.180144 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.180185 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:06:26.180271 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.176636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33683 (local address 127.18.63.66:57881)
0808 07:06:26.176711 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.176726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.176815 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.176976 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.176977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.176979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:26.177075 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.177077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.178037 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.178038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.179389 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.179392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.180011 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.180011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.180073 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.180077 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.180116 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.180181 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":21}
I20260808 07:06:26.180621 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6109dea8e8cf4e29a7c370171b126739" candidate_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" is_pre_election: true
I20260808 07:06:26.180739 20491 raft_consensus.cc:2454] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59790674c8cc4c1e87bed699ed6d7eb1 in term 0.
I20260808 07:06:26.180820 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.176872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:59575 (local address 127.18.63.67:46373)
0808 07:06:26.176941 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.176943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.176955 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:26.177062 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.177063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.177081 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.177152 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:26.177320 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.178067 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.179773 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.180302 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.180388 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.180397 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.180705 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.180749 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:06:26.180927 20595 leader_election.cc:302] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d9850f0edd4491b2a4ebaea5cfb7cd, 59790674c8cc4c1e87bed699ed6d7eb1; no voters:
I20260808 07:06:26.181018 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.176766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46373 (local address 127.18.63.66:59575)
0808 07:06:26.176913 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.176921 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.176964 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.177159 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.177160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.177162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:26.177245 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.177247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.178143 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.178144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.179684 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.179687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.180467 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.180468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.180602 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.180607 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.180900 (+ 293us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.180950 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":110,"threads_started":1}
I20260808 07:06:26.181092 21002 raft_consensus.cc:2781] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:26.181133 21002 raft_consensus.cc:482] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:26.181183 21002 raft_consensus.cc:3037] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:26.181398 20894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6109dea8e8cf4e29a7c370171b126739" candidate_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64a89dff0e345acb9caf66fad273f2c" is_pre_election: true
I20260808 07:06:26.181522 20894 raft_consensus.cc:2454] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59790674c8cc4c1e87bed699ed6d7eb1 in term 0.
I20260808 07:06:26.182087 21002 raft_consensus.cc:504] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.182241 21002 leader_election.cc:288] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [CANDIDATE]: Term 1 election: Requested vote from peers 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683), d64a89dff0e345acb9caf66fad273f2c (127.18.63.67:46373)
I20260808 07:06:26.182403 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6109dea8e8cf4e29a7c370171b126739" candidate_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd"
I20260808 07:06:26.182488 20491 raft_consensus.cc:3037] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:26.182440 20894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6109dea8e8cf4e29a7c370171b126739" candidate_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64a89dff0e345acb9caf66fad273f2c"
I20260808 07:06:26.182555 20894 raft_consensus.cc:3037] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:26.183198 20491 raft_consensus.cc:2454] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59790674c8cc4c1e87bed699ed6d7eb1 in term 1.
I20260808 07:06:26.183311 20894 raft_consensus.cc:2454] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59790674c8cc4c1e87bed699ed6d7eb1 in term 1.
I20260808 07:06:26.183382 20595 leader_election.cc:302] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d9850f0edd4491b2a4ebaea5cfb7cd, 59790674c8cc4c1e87bed699ed6d7eb1; no voters:
I20260808 07:06:26.183463 21002 raft_consensus.cc:2781] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:26.183622 21002 raft_consensus.cc:686] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 59790674c8cc4c1e87bed699ed6d7eb1, State: Running, Role: LEADER
I20260808 07:06:26.183712 21002 consensus_queue.cc:237] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } }
I20260808 07:06:26.184221 21048 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: TxnStatusTablet state changed. Reason: New leader 59790674c8cc4c1e87bed699ed6d7eb1. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.184310 21048 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:26.184491 21050 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:06:26.184645 21050 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:06:26.184592 20278 catalog_manager.cc:5495] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66). New cstate: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:26.223419 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.220594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.18.63.66:33873)
0808 07:06:26.220662 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.220664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.220689 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:06:26.220787 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.220788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.220809 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.220876 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:26.221057 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.221720 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.222594 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.222787 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.222862 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.223065 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.223177 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.223262 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.223316 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.223336 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:26.223425 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.220531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:33873 (local address 127.0.0.1:43368)
0808 07:06:26.220637 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.220650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.220712 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.220889 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.220890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.220891 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:26.220976 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.220979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.221807 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.221809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.222499 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.222501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.222861 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.222862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.222953 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.222968 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.223246 (+ 278us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.223256 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.223301 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.223355 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 07:06:26.224695 20491 raft_consensus.cc:1261] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Refusing update from remote peer 59790674c8cc4c1e87bed699ed6d7eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:26.224821 20894 raft_consensus.cc:1261] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Refusing update from remote peer 59790674c8cc4c1e87bed699ed6d7eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:26.224990 21002 consensus_queue.cc:1035] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:26.225111 21048 consensus_queue.cc:1035] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:26.226259 21003 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: TxnStatusTablet state changed. Reason: New leader 59790674c8cc4c1e87bed699ed6d7eb1. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.226370 21003 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.226805 20999 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: TxnStatusTablet state changed. Reason: New leader 59790674c8cc4c1e87bed699ed6d7eb1. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.226887 20999 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.227315 21048 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.227339 21002 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.227486 21002 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:26.227423 21048 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 59790674c8cc4c1e87bed699ed6d7eb1: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:26.228277 21003 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.228360 21003 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P d64a89dff0e345acb9caf66fad273f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.228739 20999 tablet_replica.cc:439] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 } } peers { permanent_uuid: "d64a89dff0e345acb9caf66fad273f2c" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46373 } } peers { permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 } } }
I20260808 07:06:26.228835 20999 tablet_replica.cc:442] T 6109dea8e8cf4e29a7c370171b126739 P 40d9850f0edd4491b2a4ebaea5cfb7cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:26.234946 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.232299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:45703 (local address 127.18.63.66:33873)
0808 07:06:26.232345 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.232347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.232376 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:06:26.232508 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.232509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.232520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.232603 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:26.232732 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.233249 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.234143 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.234374 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.234466 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.234662 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.234700 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.234788 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.234855 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.234876 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:26.235001 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.232229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:33873 (local address 127.18.63.67:45703)
0808 07:06:26.232322 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.232337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.232394 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.232607 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.232608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.232609 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:26.232660 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.232662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.233372 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.233373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.234029 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.234031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.234466 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.234467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.234534 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.234558 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.234791 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.234794 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.234851 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.234909 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:06:26.237455 20685 raft_consensus.cc:1261] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Refusing update from remote peer d64a89dff0e345acb9caf66fad273f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:26.237636 20491 raft_consensus.cc:1261] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Refusing update from remote peer d64a89dff0e345acb9caf66fad273f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:26.237696 21005 consensus_queue.cc:1035] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "59790674c8cc4c1e87bed699ed6d7eb1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:26.237802 21003 consensus_queue.cc:1035] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "40d9850f0edd4491b2a4ebaea5cfb7cd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:26.239713 21008 mvcc.cc:205] Tried to move back new op lower bound from 7316163732426739712 to 7316163731426934784. Current Snapshot: MvccSnapshot[applied={T|T < 7316163732426739712}]
I20260808 07:06:26.253022 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.249887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:59289 (local address 127.18.63.67:46373)
0808 07:06:26.249938 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.249940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.250136 (+ 196us) server_negotiation.cc:407] Connection header received
0808 07:06:26.250166 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.250167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.250179 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.250236 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:26.250391 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.251147 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.252314 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.252524 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.252587 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.252777 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.252808 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.252846 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.252925 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.252946 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:26.253356 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.249810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46373 (local address 127.18.63.66:59289)
0808 07:06:26.250000 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:26.250012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:26.250085 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:26.250244 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:26.250245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:26.250246 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:26.250316 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.250318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.251250 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.251251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.252224 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:26.252227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.252606 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.252607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:26.252668 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.252685 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:26.252868 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:26.252870 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:26.252916 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:06:26.252962 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260808 07:06:26.264779 20279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:26.264911 20279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:26.265708 20279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7c96547986a4e028cecd8359be1989e: Sending DeleteTablet for 3 replicas of tablet 2e6108675f2947b786e39efec76cf991
I20260808 07:06:26.266047 20866 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6108675f2947b786e39efec76cf991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:26 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:52646
I20260808 07:06:26.266067 20463 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6108675f2947b786e39efec76cf991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:26 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37804
I20260808 07:06:26.266170 21026 tablet_replica.cc:330] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: stopping tablet replica
I20260808 07:06:26.266213 21026 raft_consensus.cc:2229] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:26.266249 21026 raft_consensus.cc:2258] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:26.266372 20656 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e6108675f2947b786e39efec76cf991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:26 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:43360
I20260808 07:06:26.266482 21021 tablet_replica.cc:330] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: stopping tablet replica
I20260808 07:06:26.266562 21021 raft_consensus.cc:2229] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:26.266608 21021 raft_consensus.cc:2258] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:26.266667 21023 tablet_replica.cc:330] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: stopping tablet replica
I20260808 07:06:26.266700 21026 ts_tablet_manager.cc:1905] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:26.266738 21023 raft_consensus.cc:2229] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:26.266798 21023 raft_consensus.cc:2258] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:26.266927 21021 ts_tablet_manager.cc:1905] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:26.267328 21023 ts_tablet_manager.cc:1905] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:26.268587 21026 ts_tablet_manager.cc:1918] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:26.268638 21026 log.cc:1196] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/2e6108675f2947b786e39efec76cf991
I20260808 07:06:26.268769 21021 ts_tablet_manager.cc:1918] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:26.268831 21021 log.cc:1196] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/2e6108675f2947b786e39efec76cf991
I20260808 07:06:26.268882 21026 ts_tablet_manager.cc:1939] T 2e6108675f2947b786e39efec76cf991 P 40d9850f0edd4491b2a4ebaea5cfb7cd: Deleting consensus metadata
I20260808 07:06:26.269047 21023 ts_tablet_manager.cc:1918] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:26.269095 21023 log.cc:1196] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/2e6108675f2947b786e39efec76cf991
I20260808 07:06:26.269088 21021 ts_tablet_manager.cc:1939] T 2e6108675f2947b786e39efec76cf991 P 59790674c8cc4c1e87bed699ed6d7eb1: Deleting consensus metadata
I20260808 07:06:26.269405 20257 catalog_manager.cc:4841] TS 40d9850f0edd4491b2a4ebaea5cfb7cd (127.18.63.65:33683): tablet 2e6108675f2947b786e39efec76cf991 (table test-table [id=5bad40b105f74038a165d86e095aa6a9]) successfully deleted
I20260808 07:06:26.269309 21023 ts_tablet_manager.cc:1939] T 2e6108675f2947b786e39efec76cf991 P d64a89dff0e345acb9caf66fad273f2c: Deleting consensus metadata
I20260808 07:06:26.269580 20257 catalog_manager.cc:4841] TS 59790674c8cc4c1e87bed699ed6d7eb1 (127.18.63.66:33873): tablet 2e6108675f2947b786e39efec76cf991 (table test-table [id=5bad40b105f74038a165d86e095aa6a9]) successfully deleted
I20260808 07:06:26.269874 20257 catalog_manager.cc:4841] TS d64a89dff0e345acb9caf66fad273f2c (127.18.63.67:46373): tablet 2e6108675f2947b786e39efec76cf991 (table test-table [id=5bad40b105f74038a165d86e095aa6a9]) successfully deleted
Aug 08 07:06:26 dist-test-slave-bw88 krb5kdc[20216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172785, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:26.280105 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.269764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.18.63.65:33683)
0808 07:06:26.269808 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.269811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.270173 (+ 362us) server_negotiation.cc:407] Connection header received
0808 07:06:26.270243 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.270244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.270253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.270301 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:26.271108 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.271578 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.272729 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.273064 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.273132 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.276773 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:26.276785 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:26.276785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:26.276793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:26.278830 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.279201 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.279205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.279205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.279273 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.279503 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.279504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.279505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.279665 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:26.279824 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.279893 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.279996 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:06:26.280481 20542 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37816
I20260808 07:06:26.288108 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:26.281625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.18.63.126:34601)
0808 07:06:26.281682 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:26.281684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:26.281694 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:26.281771 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:26.281772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:26.281783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:26.281832 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:26.282381 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.282917 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.283801 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:26.284021 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:26.284096 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:26.285420 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:26.285422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:26.285422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:26.285429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:26.286944 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.287297 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.287299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.287299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.287360 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:26.287586 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:26.287587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:26.287587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:26.287702 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:26.287808 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:26.287920 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:06:26.288001 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:06:26.288458 20279 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41128
I20260808 07:06:26.289105 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 20377
I20260808 07:06:26.313267 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 20569
I20260808 07:06:26.322731 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 20771
I20260808 07:06:26.332224 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 20233
2026-08-08T07:06:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2470 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:26 dist-test-slave-bw88 krb5kdc[21077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:26 dist-test-slave-bw88 krb5kdc[21077](info): set up 2 sockets
Aug 08 07:06:26 dist-test-slave-bw88 krb5kdc[21077](info): commencing operation
krb5kdc: starting...
W20260808 07:06:27.527684 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:27 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:27.649243 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:34633
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:34633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:27.733418 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:27.733716 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:27.733757 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:27.734889 21094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:27.734951 21094 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:06:27.734972 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:27.734993 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:27.735023 21094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:27.735046 21094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:27.736531 21094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:34633
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:34633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:27.736805 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:27.737039 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:27.741962 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.741967 21102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.742002 21100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:27.742274 21094 server_base.cc:1034] running on GCE node
I20260808 07:06:27.742503 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:27.742792 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:27.743965 21094 hybrid_clock.cc:648] HybridClock initialized: now 1786172787743960 us; error 38 us; skew 500 ppm
Aug 08 07:06:27 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:27.746505 21094 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:27.746881 21094 webserver.cc:460] Webserver started at http://127.18.63.126:46731/ using document root <none> and password file <none>
I20260808 07:06:27.747092 21094 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:27.747133 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:27.747220 21094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:27.748206 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "e1cbbe034cfa41c991e07c58d1746cb9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "97b0e8ebb52ae51ff0b8d8a63f3094c7"
server_key_iv: "080e92b699ddfd18a6fdbf401a0e00da"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.748507 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "e1cbbe034cfa41c991e07c58d1746cb9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "97b0e8ebb52ae51ff0b8d8a63f3094c7"
server_key_iv: "080e92b699ddfd18a6fdbf401a0e00da"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.749886 21094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:27.750607 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:27.750811 21094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:27.750898 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "e1cbbe034cfa41c991e07c58d1746cb9"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "97b0e8ebb52ae51ff0b8d8a63f3094c7"
server_key_iv: "080e92b699ddfd18a6fdbf401a0e00da"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.750954 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:27.760023 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:27.762277 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:27.762439 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:27.767256 21094 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:34633
I20260808 07:06:27.767355 21203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:34633 every 8 connection(s)
I20260808 07:06:27.767668 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:27.768399 21205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:27.770852 21205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Bootstrap starting.
I20260808 07:06:27.771458 21205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:27.771778 21205 log.cc:824] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:27.772527 21205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: No bootstrap required, opened a new log
I20260808 07:06:27.774214 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 21094
I20260808 07:06:27.774290 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:27.774463 18685 external_mini_cluster.cc:1428] Setting key bd9ac2c19f00cf35da92f28c151abeed
I20260808 07:06:27.775179 21205 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } }
I20260808 07:06:27.775372 21205 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:27.775409 21205 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1cbbe034cfa41c991e07c58d1746cb9, State: Initialized, Role: FOLLOWER
I20260808 07:06:27.775552 21205 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } }
I20260808 07:06:27.775648 21205 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:27.775683 21205 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:27.775731 21205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:27.776305 21205 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } }
I20260808 07:06:27.776413 21205 leader_election.cc:302] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1cbbe034cfa41c991e07c58d1746cb9; no voters:
I20260808 07:06:27.776575 21205 leader_election.cc:288] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:27.776659 21212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:27.776808 21212 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [term 1 LEADER]: Becoming Leader. State: Replica: e1cbbe034cfa41c991e07c58d1746cb9, State: Running, Role: LEADER
I20260808 07:06:27.776844 21205 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:27.776892 21212 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } }
I20260808 07:06:27.777228 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } } }
I20260808 07:06:27.777328 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:27.777218 21212 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1cbbe034cfa41c991e07c58d1746cb9. Latest consensus state: current_term: 1 leader_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cbbe034cfa41c991e07c58d1746cb9" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 34633 } } }
I20260808 07:06:27.777554 21212 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9 [sys.catalog]: This master's current role is: LEADER
W20260808 07:06:27.778703 21235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:27.778761 21235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:27.778817 21220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:27.779022 21220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:27.780539 21220 catalog_manager.cc:1347] Generated new cluster ID: 84ddab4c1a454bfb8102fdbd113c13e3
I20260808 07:06:27.780597 21220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:27 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:27.786059 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.775050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.18.63.126:34633)
0808 07:06:27.775253 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:27.775263 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:27.775297 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:06:27.775423 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:27.775430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:27.775487 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:27.775629 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:27.776305 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.777775 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.778871 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.779266 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.779395 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:27.782329 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:27.782333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:27.782334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:27.782346 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:27.784325 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.784736 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.784739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.784739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.784859 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.785107 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.785108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.785108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.785432 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:27.785607 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:27.785715 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 07:06:27.785848 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260808 07:06:27.795789 21220 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:27.796335 21220 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:27.800014 21220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Generated new TSK 0
I20260808 07:06:27.800215 21220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:27.834021 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34633
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:27.924082 21240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:27.924333 21240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:27.924372 21240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:27.925614 21240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:27.925693 21240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:27.925729 21240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:06:27.925762 21240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:27.925797 21240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:27.927348 21240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:27.927619 21240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:27.927867 21240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:27.933226 21240 server_base.cc:1034] running on GCE node
W20260808 07:06:27.933197 21246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.933269 21251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:27.933190 21248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:27.933586 21240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:27.933835 21240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:27.935002 21240 hybrid_clock.cc:648] HybridClock initialized: now 1786172787934996 us; error 40 us; skew 500 ppm
Aug 08 07:06:27 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:27.937640 21240 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:27.938053 21240 webserver.cc:460] Webserver started at http://127.18.63.65:34187/ using document root <none> and password file <none>
I20260808 07:06:27.938313 21240 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:27.938385 21240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:27.938540 21240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:27.939664 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "ebedae7b68614ec1910376f9aa9f6db2"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "3487df19d3d99641b2cecd8c3bf4c90e"
server_key_iv: "f070f83b25655233ac1271b852e63176"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.940032 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "ebedae7b68614ec1910376f9aa9f6db2"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "3487df19d3d99641b2cecd8c3bf4c90e"
server_key_iv: "f070f83b25655233ac1271b852e63176"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.941434 21240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:27.942188 21266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:27.942385 21240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:27.942479 21240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "ebedae7b68614ec1910376f9aa9f6db2"
format_stamp: "Formatted at 2026-08-08 07:06:27 on dist-test-slave-bw88"
server_key: "3487df19d3d99641b2cecd8c3bf4c90e"
server_key_iv: "f070f83b25655233ac1271b852e63176"
server_key_version: "encryptionkey@0"
I20260808 07:06:27.942562 21240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:27.968951 21240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:27.971494 21240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:27.971679 21240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:27.972229 21240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:27.972296 21240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:27.972347 21240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:27.972373 21240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:27.979844 21240 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:33655
I20260808 07:06:27.979955 21417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:33655 every 8 connection(s)
Aug 08 07:06:27 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:27.980259 21240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:27.983291 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.972830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:49457 (local address 127.18.63.126:34633)
0808 07:06:27.972888 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:27.972890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:27.973123 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:06:27.973755 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:27.973756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:27.973788 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:27.973837 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:27.974750 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.975642 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.977298 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.977661 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.977749 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:27.980579 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:27.980581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:27.980581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:27.980590 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:27.982112 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.982487 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.982488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.982489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.982541 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.982762 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.982763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.982763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.982862 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:27.982914 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:27.983156 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:06:27.983226 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:06:27.983433 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.972805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.65:49457)
0808 07:06:27.972998 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:27.973036 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:27.973634 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:27.973921 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:27.973924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:27.974340 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:27.974625 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:27.974633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.975752 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.975753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:27.977214 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:27.977218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.977785 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.977787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:27.977974 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:27.978535 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:27.978546 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:27.980464 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:27.982189 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:27.982193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:27.982195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:27.982416 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:27.982600 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:27.982600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:27.982601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:27.982707 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:27.982917 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:27.982922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:27.983037 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 07:06:27.983120 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:06:27.983245 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260808 07:06:27.983991 21418 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34633
I20260808 07:06:27.984078 21418 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:27.984280 21418 heartbeater.cc:511] Master 127.18.63.126:34633 requested a full tablet report, sending...
I20260808 07:06:27.984876 21140 ts_manager.cc:194] Registered new tserver with Master: ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655)
I20260808 07:06:27.985716 21140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:49457
I20260808 07:06:27.990136 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 21240
I20260808 07:06:27.990262 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:27.990451 18685 external_mini_cluster.cc:1428] Setting key 1eadf533f9f3bc6b98e4e7a611dee324
I20260808 07:06:27.990986 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.985136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.65:47235)
0808 07:06:27.985305 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:27.985316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:27.985400 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:27.985545 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:27.985546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:27.985727 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:27.985776 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:27.985777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.986474 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.986475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:27.986995 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:27.986996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.987382 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.987383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:27.987463 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:27.987886 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:27.987891 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:27.988170 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:27.989822 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:27.989823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:27.989824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:27.990074 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:27.990337 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:27.990337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:27.990338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:27.990408 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:27.990687 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:27.990688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:27.990752 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 07:06:27.990830 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:06:27.990922 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260808 07:06:27.991047 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:27.985228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:47235 (local address 127.18.63.126:34633)
0808 07:06:27.985290 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:27.985292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:27.985387 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:06:27.985459 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:27.985460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:27.985471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:27.985527 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:27.985856 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.986371 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.987080 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:27.987293 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:27.987380 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:27.988278 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:27.988279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:27.988279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:27.988286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:27.989720 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.990171 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.990172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.990172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.990258 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:27.990487 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:27.990488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:27.990488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:27.990602 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:27.990698 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:27.990873 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 07:06:27.990956 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:28.016292 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34633
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:28.101410 21430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:28.101622 21430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:28.101647 21430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:28.102754 21430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:28.102792 21430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:28.102813 21430 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:06:28.102824 21430 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:28.102841 21430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:28.104244 21430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:28.104440 21430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:28.104624 21430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:28.110308 21435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:28.110314 21436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:28.110664 21430 server_base.cc:1034] running on GCE node
W20260808 07:06:28.110792 21438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:28.111086 21430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:28.111388 21430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:28.112579 21430 hybrid_clock.cc:648] HybridClock initialized: now 1786172788112567 us; error 30 us; skew 500 ppm
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:28.115278 21430 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:28.115696 21430 webserver.cc:460] Webserver started at http://127.18.63.66:45287/ using document root <none> and password file <none>
I20260808 07:06:28.115931 21430 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:28.115980 21430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:28.116122 21430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:28.117143 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "18e5f409d3be45f59ceea4a2e368dceb"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "e3ce22af7f56c7cecd5b9671c533fa1f"
server_key_iv: "02fee0cdb04abdfff6bdd6212f37efaf"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.117534 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "18e5f409d3be45f59ceea4a2e368dceb"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "e3ce22af7f56c7cecd5b9671c533fa1f"
server_key_iv: "02fee0cdb04abdfff6bdd6212f37efaf"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.118891 21430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:28.119645 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.119850 21430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:28.119966 21430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "18e5f409d3be45f59ceea4a2e368dceb"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "e3ce22af7f56c7cecd5b9671c533fa1f"
server_key_iv: "02fee0cdb04abdfff6bdd6212f37efaf"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.120038 21430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:28.136247 21430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:28.138222 21430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:28.138386 21430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:28.138893 21430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:28.138937 21430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.138962 21430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:28.138974 21430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:28.146876 21430 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:36239
I20260808 07:06:28.146908 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:36239 every 8 connection(s)
I20260808 07:06:28.147338 21430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:28.149091 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.139414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:44903 (local address 127.18.63.126:34633)
0808 07:06:28.139457 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.139459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.139693 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:06:28.140216 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.140217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.140231 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.140295 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.141049 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.141813 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.142837 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.143086 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.143150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.146078 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.146079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.146080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.146089 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.147915 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.148354 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.148355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.148355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.148410 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.148609 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.148610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.148610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.148705 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.148763 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.148965 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.149029 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:06:28.149303 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.139397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.66:44903)
0808 07:06:28.139572 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.139606 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.140100 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.140368 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.140371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.140681 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:28.140957 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.140964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.141931 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.141932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.142742 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.142745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.143162 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.143163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.143323 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.143911 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:28.143920 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:28.145976 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:28.147986 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.147991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.147992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.148295 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.148470 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.148471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.148471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.148569 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.148766 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:28.148771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:28.148876 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.148952 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.149067 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":113,"threads_started":1}
I20260808 07:06:28.149896 21611 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34633
I20260808 07:06:28.149981 21611 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:28.150163 21611 heartbeater.cc:511] Master 127.18.63.126:34633 requested a full tablet report, sending...
I20260808 07:06:28.150614 21140 ts_manager.cc:194] Registered new tserver with Master: 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239)
I20260808 07:06:28.151103 21140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:44903
I20260808 07:06:28.151746 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 21430
I20260808 07:06:28.151844 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:28.152025 18685 external_mini_cluster.cc:1428] Setting key c9e40885557cede4e771bc5bef19d035
I20260808 07:06:28.156859 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.150990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.66:46309)
0808 07:06:28.151147 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.151157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.151216 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.151350 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.151351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.151543 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:28.151622 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.151624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.152400 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.152400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.152936 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.152938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.153376 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.153378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.153453 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.153877 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:28.153881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:28.154190 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:28.155794 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.155794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.155795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.155967 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.156176 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.156177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.156177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.156238 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.156528 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:28.156528 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:28.156575 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.156662 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.156764 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260808 07:06:28.156862 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.151074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:46309 (local address 127.18.63.126:34633)
0808 07:06:28.151118 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.151120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.151206 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:06:28.151262 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.151263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.151274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.151336 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.151721 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.152284 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.153034 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.153282 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.153377 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.154291 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.154293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.154293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.154300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.155687 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.156041 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.156042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.156042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.156097 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.156310 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.156311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.156312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.156454 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.156534 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.156687 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.156782 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:28.178416 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:34633
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:28.262459 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:28.262688 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:28.262712 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:28.263890 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:28.263931 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:28.263952 21623 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 07:06:28.263963 21623 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:28.263980 21623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:28.265381 21623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:34633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:28.265563 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:28.265753 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:28.270824 21631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:28.270789 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:28.270789 21632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:28.270947 21623 server_base.cc:1034] running on GCE node
I20260808 07:06:28.271234 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:28.271467 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:28.272604 21623 hybrid_clock.cc:648] HybridClock initialized: now 1786172788272596 us; error 29 us; skew 500 ppm
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:28.275143 21623 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:28.275559 21623 webserver.cc:460] Webserver started at http://127.18.63.67:43699/ using document root <none> and password file <none>
I20260808 07:06:28.275781 21623 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:28.275848 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:28.276001 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:28.277117 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "6f0d6ebdc61546d593cb450476beebde"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "ed9947dfc4d8355fb17f38809f415e64"
server_key_iv: "7e2496568b2a83e9d1134505d3a43ee7"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.277591 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "6f0d6ebdc61546d593cb450476beebde"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "ed9947dfc4d8355fb17f38809f415e64"
server_key_iv: "7e2496568b2a83e9d1134505d3a43ee7"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.278942 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:28.279656 21645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.279827 21623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:28.279956 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "6f0d6ebdc61546d593cb450476beebde"
format_stamp: "Formatted at 2026-08-08 07:06:28 on dist-test-slave-bw88"
server_key: "ed9947dfc4d8355fb17f38809f415e64"
server_key_iv: "7e2496568b2a83e9d1134505d3a43ee7"
server_key_version: "encryptionkey@0"
I20260808 07:06:28.280030 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:28.306739 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:28.309219 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:28.309446 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:28.310009 21623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:28.310074 21623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.310118 21623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:28.310156 21623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.316797 21623 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:34155
I20260808 07:06:28.316918 21792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:34155 every 8 connection(s)
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:28.317202 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:28.320112 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.310528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:35041 (local address 127.18.63.126:34633)
0808 07:06:28.310593 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.310595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.310817 (+ 222us) server_negotiation.cc:407] Connection header received
0808 07:06:28.311365 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.311367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.311381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.311446 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.312349 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.313185 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.314786 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.315090 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.315176 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.317788 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.317790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.317790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.317797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.318965 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.319338 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.319339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.319339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.319391 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.319605 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.319606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.319606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.319709 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.319760 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.319989 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.320050 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:28.320354 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.310524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.67:35041)
0808 07:06:28.310691 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.310727 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.311255 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.311524 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.311527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.311920 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:28.312219 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.312227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.313324 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.313326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.314687 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.314691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.315177 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.315178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.315275 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.315819 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:28.315829 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:28.317700 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:28.319047 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.319054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.319055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.319271 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.319451 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.319451 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.319452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.319551 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.319763 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:28.319768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:28.319878 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.319959 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.320095 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
I20260808 07:06:28.320811 21794 heartbeater.cc:347] Connected to a master server at 127.18.63.126:34633
I20260808 07:06:28.320881 21794 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:28.321065 21794 heartbeater.cc:511] Master 127.18.63.126:34633 requested a full tablet report, sending...
I20260808 07:06:28.321517 21138 ts_manager.cc:194] Registered new tserver with Master: 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.322093 21138 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:35041
I20260808 07:06:28.323622 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 21623
I20260808 07:06:28.323722 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:28.323930 18685 external_mini_cluster.cc:1428] Setting key c7b36df5eef21f759b5512aab56b744e
I20260808 07:06:28.324903 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:28.327661 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.321990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:50847 (local address 127.18.63.126:34633)
0808 07:06:28.322030 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.322032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.322156 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:06:28.322254 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.322255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.322267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.322319 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.322667 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.323178 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.323997 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.324275 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.324350 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.325312 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.325313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.325314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.325321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.326572 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.326916 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.326917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.326918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.326971 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.327176 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.327177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.327178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.327283 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.327351 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.327508 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.327578 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:28.327682 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.321916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.18.63.67:50847)
0808 07:06:28.322078 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.322092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.322172 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.322335 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.322336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.322523 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:28.322571 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.322573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.323274 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.323275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.323900 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.323902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.324406 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.324407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.324482 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.324909 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:28.324913 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:28.325224 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:28.326660 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.326661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.326661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.326831 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.327045 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.327046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.327047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.327103 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.327357 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:28.327358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:28.327401 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.327484 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.327580 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:28.348726 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.342089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.18.63.126:34633)
0808 07:06:28.342146 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.342150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.342164 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:06:28.342184 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.342185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.342193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.342241 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.342787 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.343235 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.343925 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.344096 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.344144 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.346488 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.346495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.346495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.346501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.347620 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.347968 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.347969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.347969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.348027 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.348236 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.348237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.348238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.348338 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.348444 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.348543 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.348617 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:06:28.350412 21138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:28.352056 21138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:28.358403 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33655 (local address 127.0.0.1:38290)
0808 07:06:28.354684 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.354694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.354763 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.355012 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.355015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.355017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.355081 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.355083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.356039 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.356040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.357208 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.357211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.358099 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.358100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.358179 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.358194 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.358268 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.358321 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260808 07:06:28.358441 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.18.63.65:33655)
0808 07:06:28.354712 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.354716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.354828 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:06:28.354898 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.354900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.354917 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.355004 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.355224 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.355947 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.357305 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.357991 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.358100 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.358171 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.358291 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.358332 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260808 07:06:28.358831 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:34155 (local address 127.0.0.1:48696)
0808 07:06:28.354818 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.354828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.354893 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.355293 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.355294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.355295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.355329 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.355330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.356504 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.356505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.357591 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.357593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.358599 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.358600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.358661 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.358665 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.358711 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.358755 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260808 07:06:28.359107 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.18.63.67:34155)
0808 07:06:28.355080 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.355085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.355095 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:28.355132 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.355133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.355149 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.355226 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.355401 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.356422 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.357685 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.358412 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.358473 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.358559 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.358959 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.359005 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260808 07:06:28.359140 21331 tablet_service.cc:1467] Processing CreateTablet for tablet d3f204f0b6614a198cfea3258294858a (DEFAULT_TABLE table=test-table [id=45a103e22de64a749187ae9d1e767628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.359566 21331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f204f0b6614a198cfea3258294858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.359822 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.18.63.66:36239)
0808 07:06:28.354790 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.354794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.354805 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:28.354847 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.354849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.354866 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.354961 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.355191 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.356306 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.357520 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.358384 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.359086 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.359657 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.359686 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.359731 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":90,"threads_started":1}
I20260808 07:06:28.359740 21713 tablet_service.cc:1467] Processing CreateTablet for tablet d3f204f0b6614a198cfea3258294858a (DEFAULT_TABLE table=test-table [id=45a103e22de64a749187ae9d1e767628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.359974 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.354371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:36239 (local address 127.0.0.1:54182)
0808 07:06:28.354533 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.354575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.354651 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.355038 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.355039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.355042 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.355102 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.355105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.356398 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.356399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.357417 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.357420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.358773 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.358773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.358833 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.358837 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.358871 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.359549 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":48,"thread_start_us":57,"threads_started":1}
I20260808 07:06:28.360073 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f204f0b6614a198cfea3258294858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.360598 21520 tablet_service.cc:1467] Processing CreateTablet for tablet d3f204f0b6614a198cfea3258294858a (DEFAULT_TABLE table=test-table [id=45a103e22de64a749187ae9d1e767628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.360909 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3f204f0b6614a198cfea3258294858a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.363442 21825 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Bootstrap starting.
I20260808 07:06:28.363852 21826 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Bootstrap starting.
I20260808 07:06:28.364099 21827 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Bootstrap starting.
I20260808 07:06:28.364521 21825 tablet_bootstrap.cc:654] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.364567 21826 tablet_bootstrap.cc:654] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.364789 21827 tablet_bootstrap.cc:654] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.364892 21825 log.cc:824] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:28.364905 21826 log.cc:824] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:28.365124 21827 log.cc:824] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:28.365911 21826 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: No bootstrap required, opened a new log
I20260808 07:06:28.365942 21825 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: No bootstrap required, opened a new log
I20260808 07:06:28.365983 21826 ts_tablet_manager.cc:1397] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:28.366017 21825 ts_tablet_manager.cc:1397] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:28.367360 21826 raft_consensus.cc:348] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.367488 21826 raft_consensus.cc:374] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.367386 21825 raft_consensus.cc:348] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.367512 21826 raft_consensus.cc:729] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.367532 21825 raft_consensus.cc:374] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.367581 21825 raft_consensus.cc:729] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0d6ebdc61546d593cb450476beebde, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.367648 21826 consensus_queue.cc:260] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.367697 21825 consensus_queue.cc:260] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.367954 21826 ts_tablet_manager.cc:1428] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:28.367982 21825 ts_tablet_manager.cc:1428] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:28.367986 21418 heartbeater.cc:503] Master 127.18.63.126:34633 was elected leader, sending a full tablet report...
I20260808 07:06:28.368127 21794 heartbeater.cc:503] Master 127.18.63.126:34633 was elected leader, sending a full tablet report...
I20260808 07:06:28.368733 21827 tablet_bootstrap.cc:492] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: No bootstrap required, opened a new log
I20260808 07:06:28.368811 21827 ts_tablet_manager.cc:1397] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 07:06:28.370704 21827 raft_consensus.cc:348] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.370831 21827 raft_consensus.cc:374] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.370851 21827 raft_consensus.cc:729] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.370960 21827 consensus_queue.cc:260] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.371201 21827 ts_tablet_manager.cc:1428] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 07:06:28.371280 21611 heartbeater.cc:503] Master 127.18.63.126:34633 was elected leader, sending a full tablet report...
W20260808 07:06:28.398057 21612 tablet.cc:2369] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:06:28.481168 21419 tablet.cc:2369] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:28.548707 21835 raft_consensus.cc:482] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:28.548836 21835 raft_consensus.cc:504] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.549288 21835 leader_election.cc:288] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.552235 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.549587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43031 (local address 127.18.63.65:33655)
0808 07:06:28.549661 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.549664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.549677 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:28.549748 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.549749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.549760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.549821 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.550011 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.550589 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.551509 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.551873 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.551959 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.551969 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.552135 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.552176 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:06:28.552235 21465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.549513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33655 (local address 127.18.63.66:43031)
0808 07:06:28.549577 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.549592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.549680 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.549834 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.549835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.549837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.549908 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.549910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.550675 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.550676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.551413 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.551415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.551959 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.551960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.552050 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.552053 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.552122 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.552161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:06:28.552424 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.549675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:34155 (local address 127.18.63.66:50239)
0808 07:06:28.549836 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.549847 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.549908 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.550066 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.550067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.550068 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.550140 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.550141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.550879 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.550880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.551593 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.551595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.552137 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.552138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.552226 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.552229 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.552290 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.552328 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260808 07:06:28.552412 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.549810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:50239 (local address 127.18.63.67:34155)
0808 07:06:28.549859 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.549869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.549894 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:06:28.549974 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.549974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.549982 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.550034 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.550212 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.550793 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.551692 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.552053 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.552136 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.552144 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.552308 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.552348 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:28.552587 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f204f0b6614a198cfea3258294858a" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebedae7b68614ec1910376f9aa9f6db2" is_pre_election: true
I20260808 07:06:28.552757 21354 raft_consensus.cc:2454] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 0.
I20260808 07:06:28.553007 21453 leader_election.cc:302] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:28.553151 21835 raft_consensus.cc:2781] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:28.553118 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f204f0b6614a198cfea3258294858a" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde" is_pre_election: true
I20260808 07:06:28.553201 21835 raft_consensus.cc:482] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:28.553227 21835 raft_consensus.cc:3037] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.553277 21734 raft_consensus.cc:2454] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 0.
I20260808 07:06:28.554080 21835 raft_consensus.cc:504] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.554215 21835 leader_election.cc:288] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 election: Requested vote from peers ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.554380 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f204f0b6614a198cfea3258294858a" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebedae7b68614ec1910376f9aa9f6db2"
I20260808 07:06:28.554492 21354 raft_consensus.cc:3037] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.554457 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3f204f0b6614a198cfea3258294858a" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde"
I20260808 07:06:28.554577 21734 raft_consensus.cc:3037] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.555263 21354 raft_consensus.cc:2454] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 1.
I20260808 07:06:28.555393 21734 raft_consensus.cc:2454] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 1.
I20260808 07:06:28.555483 21453 leader_election.cc:302] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:28.555577 21835 raft_consensus.cc:2781] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:28.555719 21835 raft_consensus.cc:686] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 1 LEADER]: Becoming Leader. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Running, Role: LEADER
I20260808 07:06:28.555842 21835 consensus_queue.cc:237] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.556720 21140 catalog_manager.cc:5495] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66). New cstate: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } health_report { overall_health: UNKNOWN } } }
W20260808 07:06:28.567829 21795 tablet.cc:2369] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:06:28 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172787, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:28.600797 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.594206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:34633 (local address 127.0.0.1:41268)
0808 07:06:28.594407 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.594418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.594491 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.594652 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.594652 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.594956 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:28.595005 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.595007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.595640 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.595641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.596148 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.596149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.596469 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.596470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.596512 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.597036 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:28.597041 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:28.598388 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:28.599753 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.599760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.599762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.599953 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.600172 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:28.600174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:28.600174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:28.600251 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:28.600494 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:28.600500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:28.600542 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.600602 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.600703 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":127,"threads_started":1}
I20260808 07:06:28.600847 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.594273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41268 (local address 127.18.63.126:34633)
0808 07:06:28.594344 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.594347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.594486 (+ 139us) server_negotiation.cc:407] Connection header received
0808 07:06:28.594555 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.594556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.594566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.594619 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:28.595069 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.595562 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.596216 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.596401 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.596453 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.598472 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:28.598474 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:28.598474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:28.598480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:28.599682 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.600019 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.600020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.600020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.600083 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:28.600322 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:28.600322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:28.600323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:28.600421 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:28.600471 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.600629 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.600751 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":37}
I20260808 07:06:28.603149 21140 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:41268:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:06:28.603381 21140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:28.604952 21331 tablet_service.cc:1467] Processing CreateTablet for tablet a430efbb2b01466fad2d86ff3c05f5fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15cb0575e2594b8883fbb4262ca267ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:28.605132 21331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a430efbb2b01466fad2d86ff3c05f5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.605192 21520 tablet_service.cc:1467] Processing CreateTablet for tablet a430efbb2b01466fad2d86ff3c05f5fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15cb0575e2594b8883fbb4262ca267ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:28.605448 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a430efbb2b01466fad2d86ff3c05f5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.606649 21826 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: Bootstrap starting.
I20260808 07:06:28.606786 21713 tablet_service.cc:1467] Processing CreateTablet for tablet a430efbb2b01466fad2d86ff3c05f5fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15cb0575e2594b8883fbb4262ca267ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:28.606961 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a430efbb2b01466fad2d86ff3c05f5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.607066 21826 tablet_bootstrap.cc:654] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.607402 21827 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: Bootstrap starting.
I20260808 07:06:28.607847 21827 tablet_bootstrap.cc:654] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.608484 21826 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: No bootstrap required, opened a new log
I20260808 07:06:28.608541 21826 ts_tablet_manager.cc:1397] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:28.608726 21825 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: Bootstrap starting.
I20260808 07:06:28.608721 21826 raft_consensus.cc:348] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.608794 21826 raft_consensus.cc:374] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.608819 21826 raft_consensus.cc:729] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.608876 21826 consensus_queue.cc:260] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.608985 21826 ts_tablet_manager.cc:1428] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.609050 21833 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.609144 21833 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.609187 21827 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: No bootstrap required, opened a new log
I20260808 07:06:28.609227 21827 ts_tablet_manager.cc:1397] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260808 07:06:28.609431 21825 tablet_bootstrap.cc:654] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.609387 21827 raft_consensus.cc:348] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.609472 21827 raft_consensus.cc:374] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.609494 21827 raft_consensus.cc:729] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.609543 21827 consensus_queue.cc:260] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.609640 21827 ts_tablet_manager.cc:1428] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:28.609683 21835 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.609787 21835 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.610409 21825 tablet_bootstrap.cc:492] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: No bootstrap required, opened a new log
I20260808 07:06:28.610453 21825 ts_tablet_manager.cc:1397] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:28.610574 21825 raft_consensus.cc:348] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.610633 21825 raft_consensus.cc:374] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.610647 21825 raft_consensus.cc:729] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0d6ebdc61546d593cb450476beebde, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.610704 21825 consensus_queue.cc:260] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.610797 21825 ts_tablet_manager.cc:1428] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.610862 21834 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.610935 21834 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.726181 21835 raft_consensus.cc:482] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:28.726298 21835 raft_consensus.cc:504] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.726542 21835 leader_election.cc:288] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.726814 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a430efbb2b01466fad2d86ff3c05f5fc" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebedae7b68614ec1910376f9aa9f6db2" is_pre_election: true
I20260808 07:06:28.726833 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a430efbb2b01466fad2d86ff3c05f5fc" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde" is_pre_election: true
I20260808 07:06:28.726926 21354 raft_consensus.cc:2454] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 0.
I20260808 07:06:28.726956 21734 raft_consensus.cc:2454] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 0.
I20260808 07:06:28.727108 21453 leader_election.cc:302] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, 6f0d6ebdc61546d593cb450476beebde; no voters:
I20260808 07:06:28.727274 21835 raft_consensus.cc:2781] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:28.727327 21835 raft_consensus.cc:482] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:28.727344 21835 raft_consensus.cc:3037] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.728179 21835 raft_consensus.cc:504] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.728330 21835 leader_election.cc:288] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 election: Requested vote from peers ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.728549 21354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a430efbb2b01466fad2d86ff3c05f5fc" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebedae7b68614ec1910376f9aa9f6db2"
I20260808 07:06:28.728565 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a430efbb2b01466fad2d86ff3c05f5fc" candidate_uuid: "18e5f409d3be45f59ceea4a2e368dceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde"
I20260808 07:06:28.728637 21354 raft_consensus.cc:3037] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.728665 21734 raft_consensus.cc:3037] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.729298 21734 raft_consensus.cc:2454] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 1.
I20260808 07:06:28.729421 21354 raft_consensus.cc:2454] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e5f409d3be45f59ceea4a2e368dceb in term 1.
I20260808 07:06:28.729511 21453 leader_election.cc:302] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, 6f0d6ebdc61546d593cb450476beebde; no voters:
I20260808 07:06:28.729602 21835 raft_consensus.cc:2781] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:28.729660 21835 raft_consensus.cc:686] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [term 1 LEADER]: Becoming Leader. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Running, Role: LEADER
I20260808 07:06:28.729707 21835 consensus_queue.cc:237] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.729998 21837 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: TxnStatusTablet state changed. Reason: New leader 18e5f409d3be45f59ceea4a2e368dceb. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.730103 21837 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:28.730290 21855 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:06:28.730432 21855 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:06:28.730420 21140 catalog_manager.cc:5495] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66). New cstate: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:28.755506 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.752916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.18.63.66:36239)
0808 07:06:28.752984 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.752987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.752996 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:28.753031 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.753032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.753051 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.753150 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.753287 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.753806 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.754659 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.754869 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.754945 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.755153 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.755256 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.755347 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.755391 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.755412 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:06:28.755514 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.752800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:36239 (local address 127.0.0.1:54194)
0808 07:06:28.752871 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.752883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.752951 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.753157 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.753158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.753159 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:28.753229 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.753232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.753901 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.753902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.754555 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.754557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.754947 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.754948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.755019 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.755038 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.755326 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.755337 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.755388 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.755439 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:06:28.757468 21734 raft_consensus.cc:1261] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Refusing update from remote peer 18e5f409d3be45f59ceea4a2e368dceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.757507 21354 raft_consensus.cc:1261] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Refusing update from remote peer 18e5f409d3be45f59ceea4a2e368dceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.757899 21837 consensus_queue.cc:1035] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.758060 21835 consensus_queue.cc:1035] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.759379 21833 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: TxnStatusTablet state changed. Reason: New leader 18e5f409d3be45f59ceea4a2e368dceb. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.759398 21834 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: TxnStatusTablet state changed. Reason: New leader 18e5f409d3be45f59ceea4a2e368dceb. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.759482 21833 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.759507 21834 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.760444 21835 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.760532 21835 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:28.760706 21833 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.760795 21833 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P ebedae7b68614ec1910376f9aa9f6db2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.760996 21860 mvcc.cc:205] Tried to move back new op lower bound from 7316163742747672576 to 7316163742637293568. Current Snapshot: MvccSnapshot[applied={T|T < 7316163742747672576}]
I20260808 07:06:28.761109 21858 mvcc.cc:205] Tried to move back new op lower bound from 7316163742747672576 to 7316163742637293568. Current Snapshot: MvccSnapshot[applied={T|T < 7316163742747672576}]
I20260808 07:06:28.761809 21837 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.761910 21837 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 18e5f409d3be45f59ceea4a2e368dceb: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:28.762389 21859 mvcc.cc:205] Tried to move back new op lower bound from 7316163742747672576 to 7316163742637293568. Current Snapshot: MvccSnapshot[applied={T|T < 7316163742747672576}]
I20260808 07:06:28.762365 21834 tablet_replica.cc:439] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18e5f409d3be45f59ceea4a2e368dceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } } }
I20260808 07:06:28.762799 21834 tablet_replica.cc:442] T a430efbb2b01466fad2d86ff3c05f5fc P 6f0d6ebdc61546d593cb450476beebde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:28.766649 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.763903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.18.63.66:36239)
0808 07:06:28.763965 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.763967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.763977 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:28.764001 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.764002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.764013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.764067 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.764230 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.764951 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.765924 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.766137 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.766216 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.766386 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.766419 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.766469 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.766537 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.766557 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:28.774792 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.772045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:42795 (local address 127.18.63.66:36239)
0808 07:06:28.772088 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.772089 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.772214 (+ 125us) server_negotiation.cc:407] Connection header received
0808 07:06:28.772275 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.772275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.772282 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.772326 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.772492 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.772906 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.774093 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.774275 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.774330 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.774554 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.774578 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.774616 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.774712 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.774727 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:28.774928 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.771946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:36239 (local address 127.18.63.66:42795)
0808 07:06:28.772121 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.772133 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.772210 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.772332 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.772333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.772335 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:28.772427 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.772428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.772984 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.772985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.773997 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.774000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.774355 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.774356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.774432 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.774452 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.774644 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.774646 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.774705 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.774752 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 07:06:28.777467 21354 raft_consensus.cc:1261] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Refusing update from remote peer 18e5f409d3be45f59ceea4a2e368dceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.777465 21734 raft_consensus.cc:1261] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Refusing update from remote peer 18e5f409d3be45f59ceea4a2e368dceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.777894 21867 consensus_queue.cc:1035] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.778056 21835 consensus_queue.cc:1035] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.790201 21140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:28.790307 21140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:28.791050 21140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Sending DeleteTablet for 3 replicas of tablet d3f204f0b6614a198cfea3258294858a
I20260808 07:06:28.791396 21331 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f204f0b6614a198cfea3258294858a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:28 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38290
I20260808 07:06:28.791451 21713 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f204f0b6614a198cfea3258294858a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:28 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:48696
I20260808 07:06:28.791728 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet d3f204f0b6614a198cfea3258294858a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:28 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:54182
I20260808 07:06:28.791744 21140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:28.791882 21140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:28.791957 21881 tablet_replica.cc:330] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: stopping tablet replica
I20260808 07:06:28.792058 21881 raft_consensus.cc:2229] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:28.792158 21881 raft_consensus.cc:2258] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:28.792691 21881 ts_tablet_manager.cc:1905] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:28.793162 21519 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea763688cf45889433424f384cddcf (DEFAULT_TABLE table=test-table [id=9b229556bed440f691a0dea985e43fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.793211 21331 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea763688cf45889433424f384cddcf (DEFAULT_TABLE table=test-table [id=9b229556bed440f691a0dea985e43fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.793268 21713 tablet_service.cc:1467] Processing CreateTablet for tablet 0dea763688cf45889433424f384cddcf (DEFAULT_TABLE table=test-table [id=9b229556bed440f691a0dea985e43fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:28.793315 21331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea763688cf45889433424f384cddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.793325 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea763688cf45889433424f384cddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.793442 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dea763688cf45889433424f384cddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:28.794174 21880 tablet_replica.cc:330] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: stopping tablet replica
I20260808 07:06:28.794292 21880 raft_consensus.cc:2229] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:28.794392 21880 raft_consensus.cc:2258] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:28.794927 21826 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Bootstrap starting.
I20260808 07:06:28.794981 21827 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Bootstrap starting.
I20260808 07:06:28.795003 21825 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Bootstrap starting.
I20260808 07:06:28.795368 21826 tablet_bootstrap.cc:654] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.795388 21827 tablet_bootstrap.cc:654] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.795411 21825 tablet_bootstrap.cc:654] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:28.796586 21825 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: No bootstrap required, opened a new log
I20260808 07:06:28.796655 21825 ts_tablet_manager.cc:1397] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:28.796802 21825 raft_consensus.cc:348] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.796880 21825 raft_consensus.cc:374] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.796900 21825 raft_consensus.cc:729] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0d6ebdc61546d593cb450476beebde, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.796940 21825 consensus_queue.cc:260] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.797050 21825 ts_tablet_manager.cc:1428] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:28.797418 21881 ts_tablet_manager.cc:1918] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:28.797487 21881 log.cc:1196] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/d3f204f0b6614a198cfea3258294858a
I20260808 07:06:28.797789 21827 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: No bootstrap required, opened a new log
I20260808 07:06:28.797832 21827 ts_tablet_manager.cc:1397] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:06:28.797974 21827 raft_consensus.cc:348] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.798048 21827 raft_consensus.cc:374] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.798071 21827 raft_consensus.cc:729] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.798115 21827 consensus_queue.cc:260] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.798238 21827 ts_tablet_manager.cc:1428] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:28.798354 21826 tablet_bootstrap.cc:492] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: No bootstrap required, opened a new log
I20260808 07:06:28.798394 21826 ts_tablet_manager.cc:1397] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:06:28.798544 21826 raft_consensus.cc:348] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.798641 21826 raft_consensus.cc:374] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:28.798671 21826 raft_consensus.cc:729] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Initialized, Role: FOLLOWER
I20260808 07:06:28.798717 21826 consensus_queue.cc:260] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.798808 21881 ts_tablet_manager.cc:1939] T d3f204f0b6614a198cfea3258294858a P 18e5f409d3be45f59ceea4a2e368dceb: Deleting consensus metadata
I20260808 07:06:28.798828 21826 ts_tablet_manager.cc:1428] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:28.799178 21114 catalog_manager.cc:4841] TS 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239): tablet d3f204f0b6614a198cfea3258294858a (table test-table [id=45a103e22de64a749187ae9d1e767628]) successfully deleted
I20260808 07:06:28.801487 21879 tablet_replica.cc:330] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: stopping tablet replica
I20260808 07:06:28.801618 21879 raft_consensus.cc:2229] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:28.801683 21879 pending_rounds.cc:62] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Trying to abort 2 pending ops.
I20260808 07:06:28.801728 21880 ts_tablet_manager.cc:1905] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:28.801712 21879 pending_rounds.cc:68] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316163742830632960 op_type: PARTICIPANT_OP participant_request { tablet_id: "d3f204f0b6614a198cfea3258294858a" op { type: BEGIN_TXN txn_id: 0 } }
I20260808 07:06:28.801826 21879 pending_rounds.cc:68] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316163741925289984 op_type: NO_OP noop_request { }
I20260808 07:06:28.801872 21879 raft_consensus.cc:2866] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 07:06:28.801916 21879 raft_consensus.cc:2258] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:28.802352 21879 ts_tablet_manager.cc:1905] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:28.803256 21880 ts_tablet_manager.cc:1918] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:28.803347 21880 log.cc:1196] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/d3f204f0b6614a198cfea3258294858a
I20260808 07:06:28.803686 21880 ts_tablet_manager.cc:1939] T d3f204f0b6614a198cfea3258294858a P ebedae7b68614ec1910376f9aa9f6db2: Deleting consensus metadata
I20260808 07:06:28.804067 21879 ts_tablet_manager.cc:1918] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:28.804145 21879 log.cc:1196] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/d3f204f0b6614a198cfea3258294858a
I20260808 07:06:28.804168 21112 catalog_manager.cc:4841] TS ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655): tablet d3f204f0b6614a198cfea3258294858a (table test-table [id=45a103e22de64a749187ae9d1e767628]) successfully deleted
I20260808 07:06:28.804432 21879 ts_tablet_manager.cc:1939] T d3f204f0b6614a198cfea3258294858a P 6f0d6ebdc61546d593cb450476beebde: Deleting consensus metadata
I20260808 07:06:28.804860 21112 catalog_manager.cc:4841] TS 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155): tablet d3f204f0b6614a198cfea3258294858a (table test-table [id=45a103e22de64a749187ae9d1e767628]) successfully deleted
I20260808 07:06:28.898620 21833 raft_consensus.cc:482] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:28.898756 21833 raft_consensus.cc:504] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.899123 21833 leader_election.cc:288] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.902717 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.899334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:34155 (local address 127.18.63.65:46649)
0808 07:06:28.899531 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.899541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.899626 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.899824 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.899825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.899828 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.899914 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.899916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.900841 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.900842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.901796 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.901799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.902420 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.902421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.902507 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.902512 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.902571 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.902612 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":42,"thread_start_us":86,"threads_started":1}
I20260808 07:06:28.902714 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.899465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46649 (local address 127.18.63.67:34155)
0808 07:06:28.899521 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.899523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.899601 (+ 78us) server_negotiation.cc:407] Connection header received
0808 07:06:28.899744 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.899746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.899757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.899812 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.900050 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.900746 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.901893 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.902326 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.902403 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.902412 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.902593 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.902640 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:28.902989 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea763688cf45889433424f384cddcf" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde" is_pre_election: true
I20260808 07:06:28.903093 21734 raft_consensus.cc:2454] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:28.903256 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.899562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:45207 (local address 127.18.63.66:36239)
0808 07:06:28.899602 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.899604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.899612 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:28.899643 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.899644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.899653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.899723 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:28.899948 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.900728 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.902219 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.902722 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.902795 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.902806 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.903144 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.903191 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:28.903299 21268 leader_election.cc:302] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f0d6ebdc61546d593cb450476beebde, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:28.903471 21833 raft_consensus.cc:2781] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:28.903519 21833 raft_consensus.cc:482] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:28.903563 21833 raft_consensus.cc:3037] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.903658 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.899334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:36239 (local address 127.18.63.65:45207)
0808 07:06:28.899488 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.899501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.899573 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.899762 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.899762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.899767 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:28.899857 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.899860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.900832 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.900833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.902116 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.902119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.902962 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.902963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.903043 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.903048 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.903385 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.903429 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260808 07:06:28.904413 21833 raft_consensus.cc:504] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.904567 21833 leader_election.cc:288] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Requested vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:28.904680 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea763688cf45889433424f384cddcf" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb" is_pre_election: true
I20260808 07:06:28.904752 21548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea763688cf45889433424f384cddcf" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb"
I20260808 07:06:28.904812 21549 raft_consensus.cc:2454] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:28.904979 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dea763688cf45889433424f384cddcf" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde"
I20260808 07:06:28.905042 21734 raft_consensus.cc:3037] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.905673 21734 raft_consensus.cc:2454] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 1.
I20260808 07:06:28.905853 21268 leader_election.cc:302] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f0d6ebdc61546d593cb450476beebde, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:28.905977 21833 raft_consensus.cc:2781] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:28.906168 21833 raft_consensus.cc:686] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 1 LEADER]: Becoming Leader. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Running, Role: LEADER
I20260808 07:06:28.906251 21833 consensus_queue.cc:237] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:28.907038 21140 catalog_manager.cc:5495] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "ebedae7b68614ec1910376f9aa9f6db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:28.915580 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.913245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.18.63.65:33655)
0808 07:06:28.913301 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.913303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.913310 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:28.913360 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.913361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.913381 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.913445 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.913569 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.914029 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.914761 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.914969 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.915046 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.915235 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.915352 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.915440 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.915480 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.915504 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:28.921572 21891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.918853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:36239 (local address 127.18.63.65:44197)
0808 07:06:28.919039 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.919048 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.919105 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.919232 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.919233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.919234 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:28.919283 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.919284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.920113 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.920114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.920721 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.920725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.921185 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.921186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.921234 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.921247 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.921439 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.921441 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.921471 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.921504 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260808 07:06:28.921618 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.918921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:44197 (local address 127.18.63.66:36239)
0808 07:06:28.918964 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.918966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.919119 (+ 153us) server_negotiation.cc:407] Connection header received
0808 07:06:28.919162 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.919163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.919174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.919231 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.919322 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.920039 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.920803 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.921098 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.921182 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.921325 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.921379 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.921441 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.921530 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.921549 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:28.923705 21734 raft_consensus.cc:1261] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.923700 21549 raft_consensus.cc:3037] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:28.923918 21833 consensus_queue.cc:1035] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.924723 21549 raft_consensus.cc:1261] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:28.924981 21887 consensus_queue.cc:1035] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:28.932189 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.929457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:40961 (local address 127.18.63.65:33655)
0808 07:06:28.929530 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:28.929532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:28.929617 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:06:28.929678 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:28.929679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:28.929688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:28.929743 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:28.929855 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.930589 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.931527 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.931730 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.931807 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.931958 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.931991 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.932032 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:28.932110 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:06:28.932129 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:06:28.932224 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:28.929520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33655 (local address 127.18.63.66:40961)
0808 07:06:28.929569 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:28.929578 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:28.929629 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:28.929743 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:28.929744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:28.929744 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:28.929781 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.929783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.930684 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.930685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.931461 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:28.931464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:28.931806 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:28.931807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:28.931866 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:28.931873 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:28.932047 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:28.932049 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:28.932096 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:06:28.932151 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:06:28.932510 21860 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:06:30.462435 21140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:30.462594 21140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:30.463495 21140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Sending DeleteTablet for 3 replicas of tablet 0dea763688cf45889433424f384cddcf
I20260808 07:06:30.463840 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea763688cf45889433424f384cddcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:54182
I20260808 07:06:30.463887 21331 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea763688cf45889433424f384cddcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38290
I20260808 07:06:30.463925 21713 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dea763688cf45889433424f384cddcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:48696
I20260808 07:06:30.464061 21932 tablet_replica.cc:330] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: stopping tablet replica
I20260808 07:06:30.464058 21933 tablet_replica.cc:330] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: stopping tablet replica
I20260808 07:06:30.464109 21933 raft_consensus.cc:2229] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:30.464109 21932 raft_consensus.cc:2229] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:30.464164 21932 raft_consensus.cc:2258] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.464205 21933 raft_consensus.cc:2258] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.464324 21934 tablet_replica.cc:330] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: stopping tablet replica
I20260808 07:06:30.464392 21934 raft_consensus.cc:2229] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:30.464439 21934 raft_consensus.cc:2258] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.464530 21933 ts_tablet_manager.cc:1905] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.466089 21932 ts_tablet_manager.cc:1905] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.466269 21933 ts_tablet_manager.cc:1918] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:06:30.466380 21933 log.cc:1196] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/0dea763688cf45889433424f384cddcf
I20260808 07:06:30.466758 21933 ts_tablet_manager.cc:1939] T 0dea763688cf45889433424f384cddcf P ebedae7b68614ec1910376f9aa9f6db2: Deleting consensus metadata
I20260808 07:06:30.467182 21112 catalog_manager.cc:4841] TS ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655): tablet 0dea763688cf45889433424f384cddcf (table test-table [id=9b229556bed440f691a0dea985e43fcb]) successfully deleted
I20260808 07:06:30.467852 21932 ts_tablet_manager.cc:1918] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:06:30.467905 21932 log.cc:1196] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/0dea763688cf45889433424f384cddcf
I20260808 07:06:30.467952 21934 ts_tablet_manager.cc:1905] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.468145 21932 ts_tablet_manager.cc:1939] T 0dea763688cf45889433424f384cddcf P 18e5f409d3be45f59ceea4a2e368dceb: Deleting consensus metadata
I20260808 07:06:30.468674 21114 catalog_manager.cc:4841] TS 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239): tablet 0dea763688cf45889433424f384cddcf (table test-table [id=9b229556bed440f691a0dea985e43fcb]) successfully deleted
I20260808 07:06:30.469666 21934 ts_tablet_manager.cc:1918] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:06:30.469756 21934 log.cc:1196] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/0dea763688cf45889433424f384cddcf
I20260808 07:06:30.470067 21934 ts_tablet_manager.cc:1939] T 0dea763688cf45889433424f384cddcf P 6f0d6ebdc61546d593cb450476beebde: Deleting consensus metadata
I20260808 07:06:30.470419 21112 catalog_manager.cc:4841] TS 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155): tablet 0dea763688cf45889433424f384cddcf (table test-table [id=9b229556bed440f691a0dea985e43fcb]) successfully deleted
Aug 08 07:06:30 dist-test-slave-bw88 krb5kdc[21077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172790, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:06:30 dist-test-slave-bw88 krb5kdc[21077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172790, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:30.495524 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:30.483150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.18.63.126:34633)
0808 07:06:30.483347 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:30.483349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:30.483385 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:06:30.483466 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:30.483466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:30.483477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:30.483557 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:30.484140 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.484684 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.485463 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.485666 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.485721 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:30.493181 (+ 7460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:30.493183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:30.493183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:30.493192 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:30.494522 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:30.494846 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:30.494847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:30.494847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:30.494903 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:30.495083 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:30.495084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:30.495084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:30.495196 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:30.495288 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:30.495388 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:06:30.495452 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260808 07:06:30.497181 21140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:30.497393 21140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:30.498934 21519 tablet_service.cc:1467] Processing CreateTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (DEFAULT_TABLE table=test-table [id=6872628865924cd9a7a3ad86f26ad550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.498939 21331 tablet_service.cc:1467] Processing CreateTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (DEFAULT_TABLE table=test-table [id=6872628865924cd9a7a3ad86f26ad550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.498962 21713 tablet_service.cc:1467] Processing CreateTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (DEFAULT_TABLE table=test-table [id=6872628865924cd9a7a3ad86f26ad550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.499102 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.499105 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.499119 21331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.500259 21947 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Bootstrap starting.
I20260808 07:06:30.500731 21947 tablet_bootstrap.cc:654] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.500768 21949 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Bootstrap starting.
I20260808 07:06:30.500759 21948 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Bootstrap starting.
I20260808 07:06:30.501137 21949 tablet_bootstrap.cc:654] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.501248 21948 tablet_bootstrap.cc:654] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.501694 21947 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: No bootstrap required, opened a new log
I20260808 07:06:30.501757 21947 ts_tablet_manager.cc:1397] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:30.501902 21947 raft_consensus.cc:348] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.501986 21947 raft_consensus.cc:374] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.502002 21947 raft_consensus.cc:729] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.502063 21947 consensus_queue.cc:260] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.502081 21948 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: No bootstrap required, opened a new log
I20260808 07:06:30.502178 21947 ts_tablet_manager.cc:1428] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:30.502130 21948 ts_tablet_manager.cc:1397] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:30.502343 21948 raft_consensus.cc:348] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.502431 21948 raft_consensus.cc:374] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.502460 21948 raft_consensus.cc:729] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0d6ebdc61546d593cb450476beebde, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.502524 21948 consensus_queue.cc:260] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.502771 21948 ts_tablet_manager.cc:1428] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:30.502835 21949 tablet_bootstrap.cc:492] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: No bootstrap required, opened a new log
I20260808 07:06:30.502882 21949 ts_tablet_manager.cc:1397] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:30.502991 21949 raft_consensus.cc:348] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.503070 21949 raft_consensus.cc:374] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.503101 21949 raft_consensus.cc:729] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.503152 21949 consensus_queue.cc:260] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.503245 21949 ts_tablet_manager.cc:1428] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:30.678061 21923 raft_consensus.cc:482] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:30.678190 21923 raft_consensus.cc:504] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.678442 21923 leader_election.cc:288] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:30.678777 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65f5a4cf1cfc4e1b85e0fbdcb09f06ca" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb" is_pre_election: true
I20260808 07:06:30.678874 21549 raft_consensus.cc:2454] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:30.678785 21733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65f5a4cf1cfc4e1b85e0fbdcb09f06ca" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde" is_pre_election: true
I20260808 07:06:30.678946 21733 raft_consensus.cc:2454] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:30.679061 21270 leader_election.cc:302] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:30.679194 21923 raft_consensus.cc:2781] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:30.679245 21923 raft_consensus.cc:482] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:30.679265 21923 raft_consensus.cc:3037] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.680006 21923 raft_consensus.cc:504] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.680156 21923 leader_election.cc:288] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Requested vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:30.680327 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65f5a4cf1cfc4e1b85e0fbdcb09f06ca" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb"
I20260808 07:06:30.680367 21733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65f5a4cf1cfc4e1b85e0fbdcb09f06ca" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde"
I20260808 07:06:30.680438 21549 raft_consensus.cc:3037] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.680439 21733 raft_consensus.cc:3037] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.681124 21549 raft_consensus.cc:2454] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 1.
I20260808 07:06:30.681318 21270 leader_election.cc:302] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:30.681466 21923 raft_consensus.cc:2781] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:30.681557 21923 raft_consensus.cc:686] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 1 LEADER]: Becoming Leader. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Running, Role: LEADER
I20260808 07:06:30.681650 21923 consensus_queue.cc:237] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.681702 21733 raft_consensus.cc:2454] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 1.
I20260808 07:06:30.682314 21140 catalog_manager.cc:5495] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "ebedae7b68614ec1910376f9aa9f6db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:30.733950 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:30.731530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.18.63.65:33655)
0808 07:06:30.731704 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:30.731706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:30.731712 (+ 6us) server_negotiation.cc:407] Connection header received
0808 07:06:30.731731 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:30.731731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:30.731741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:30.731800 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:30.731960 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.732469 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.733212 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:30.733487 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:30.733540 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:30.733700 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:30.733730 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:30.733773 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:30.733818 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:06:30.733836 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260808 07:06:30.736382 21733 raft_consensus.cc:1261] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:30.736409 21549 raft_consensus.cc:1261] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:30.736691 21899 consensus_queue.cc:1035] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:30.736865 21923 consensus_queue.cc:1035] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:30.747334 21138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:30.747505 21138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:30.748159 21138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Sending DeleteTablet for 3 replicas of tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca
I20260808 07:06:30.748453 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:54182
I20260808 07:06:30.748466 21331 tablet_service.cc:1514] Processing DeleteTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38290
I20260808 07:06:30.748615 21932 tablet_replica.cc:330] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: stopping tablet replica
I20260808 07:06:30.748610 21933 tablet_replica.cc:330] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: stopping tablet replica
I20260808 07:06:30.748621 21713 tablet_service.cc:1514] Processing DeleteTablet for tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:48696
I20260808 07:06:30.748698 21933 raft_consensus.cc:2229] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:30.748703 21932 raft_consensus.cc:2229] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:30.748724 21934 tablet_replica.cc:330] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: stopping tablet replica
I20260808 07:06:30.748757 21933 raft_consensus.cc:2258] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.748761 21932 raft_consensus.cc:2258] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.748780 21934 raft_consensus.cc:2229] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:30.748836 21934 raft_consensus.cc:2258] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:30.749184 21932 ts_tablet_manager.cc:1905] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.749248 21934 ts_tablet_manager.cc:1905] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.749627 21138 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:30.749801 21138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:30.750224 21933 ts_tablet_manager.cc:1905] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:30.751212 21932 ts_tablet_manager.cc:1918] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:30.751294 21932 log.cc:1196] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/65f5a4cf1cfc4e1b85e0fbdcb09f06ca
I20260808 07:06:30.751302 21519 tablet_service.cc:1467] Processing CreateTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 (DEFAULT_TABLE table=test-table [id=5e523b34666c471aa90e049eef022800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.751451 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cd6dce9a234a8b836a9c1ebec7f210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.751609 21932 ts_tablet_manager.cc:1939] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 18e5f409d3be45f59ceea4a2e368dceb: Deleting consensus metadata
I20260808 07:06:30.751976 21114 catalog_manager.cc:4841] TS 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239): tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (table test-table [id=6872628865924cd9a7a3ad86f26ad550]) successfully deleted
I20260808 07:06:30.752225 21933 ts_tablet_manager.cc:1918] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:30.752286 21933 log.cc:1196] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/65f5a4cf1cfc4e1b85e0fbdcb09f06ca
I20260808 07:06:30.752565 21933 ts_tablet_manager.cc:1939] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P ebedae7b68614ec1910376f9aa9f6db2: Deleting consensus metadata
I20260808 07:06:30.752866 21949 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Bootstrap starting.
I20260808 07:06:30.752972 21112 catalog_manager.cc:4841] TS ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655): tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (table test-table [id=6872628865924cd9a7a3ad86f26ad550]) successfully deleted
I20260808 07:06:30.752976 21713 tablet_service.cc:1467] Processing CreateTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 (DEFAULT_TABLE table=test-table [id=5e523b34666c471aa90e049eef022800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.753109 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cd6dce9a234a8b836a9c1ebec7f210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.753271 21949 tablet_bootstrap.cc:654] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.753444 21331 tablet_service.cc:1467] Processing CreateTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 (DEFAULT_TABLE table=test-table [id=5e523b34666c471aa90e049eef022800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:30.753571 21331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06cd6dce9a234a8b836a9c1ebec7f210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:30.754544 21947 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2: Bootstrap starting.
I20260808 07:06:30.754895 21947 tablet_bootstrap.cc:654] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.755563 21948 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Bootstrap starting.
I20260808 07:06:30.755587 21947 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2: No bootstrap required, opened a new log
I20260808 07:06:30.755645 21947 ts_tablet_manager.cc:1397] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:30.755767 21947 raft_consensus.cc:348] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.755856 21947 raft_consensus.cc:374] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.755883 21947 raft_consensus.cc:729] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.755927 21947 consensus_queue.cc:260] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.756031 21947 ts_tablet_manager.cc:1428] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:30.756089 21948 tablet_bootstrap.cc:654] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:30.756803 21934 ts_tablet_manager.cc:1918] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:30.756913 21934 log.cc:1196] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/65f5a4cf1cfc4e1b85e0fbdcb09f06ca
I20260808 07:06:30.757055 21948 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: No bootstrap required, opened a new log
I20260808 07:06:30.757094 21948 ts_tablet_manager.cc:1397] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:30.757138 21934 ts_tablet_manager.cc:1939] T 65f5a4cf1cfc4e1b85e0fbdcb09f06ca P 6f0d6ebdc61546d593cb450476beebde: Deleting consensus metadata
I20260808 07:06:30.757207 21948 raft_consensus.cc:348] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.757252 21948 raft_consensus.cc:374] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.757265 21948 raft_consensus.cc:729] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0d6ebdc61546d593cb450476beebde, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.757301 21948 consensus_queue.cc:260] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.757417 21948 ts_tablet_manager.cc:1428] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:30.757556 21112 catalog_manager.cc:4841] TS 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155): tablet 65f5a4cf1cfc4e1b85e0fbdcb09f06ca (table test-table [id=6872628865924cd9a7a3ad86f26ad550]) successfully deleted
I20260808 07:06:30.758226 21949 tablet_bootstrap.cc:492] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: No bootstrap required, opened a new log
I20260808 07:06:30.758275 21949 ts_tablet_manager.cc:1397] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 07:06:30.758390 21949 raft_consensus.cc:348] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.758459 21949 raft_consensus.cc:374] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:30.758474 21949 raft_consensus.cc:729] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e5f409d3be45f59ceea4a2e368dceb, State: Initialized, Role: FOLLOWER
I20260808 07:06:30.758526 21949 consensus_queue.cc:260] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.758626 21949 ts_tablet_manager.cc:1428] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:30.805444 21923 raft_consensus.cc:482] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:30.805578 21923 raft_consensus.cc:504] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.805768 21923 leader_election.cc:288] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:30.805964 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cd6dce9a234a8b836a9c1ebec7f210" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb" is_pre_election: true
I20260808 07:06:30.806085 21549 raft_consensus.cc:2454] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:30.806028 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cd6dce9a234a8b836a9c1ebec7f210" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde" is_pre_election: true
I20260808 07:06:30.806147 21734 raft_consensus.cc:2454] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 0.
I20260808 07:06:30.806260 21270 leader_election.cc:302] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:30.806396 21923 raft_consensus.cc:2781] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:30.806452 21923 raft_consensus.cc:482] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:30.806483 21923 raft_consensus.cc:3037] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.807499 21923 raft_consensus.cc:504] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.807660 21923 leader_election.cc:288] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Requested vote from peers 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239), 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155)
I20260808 07:06:30.807837 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cd6dce9a234a8b836a9c1ebec7f210" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0d6ebdc61546d593cb450476beebde"
I20260808 07:06:30.807914 21734 raft_consensus.cc:3037] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.807886 21549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06cd6dce9a234a8b836a9c1ebec7f210" candidate_uuid: "ebedae7b68614ec1910376f9aa9f6db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e5f409d3be45f59ceea4a2e368dceb"
I20260808 07:06:30.807974 21549 raft_consensus.cc:3037] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:30.808712 21549 raft_consensus.cc:2454] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 1.
I20260808 07:06:30.808849 21734 raft_consensus.cc:2454] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebedae7b68614ec1910376f9aa9f6db2 in term 1.
I20260808 07:06:30.808876 21270 leader_election.cc:302] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e5f409d3be45f59ceea4a2e368dceb, ebedae7b68614ec1910376f9aa9f6db2; no voters:
I20260808 07:06:30.808991 21923 raft_consensus.cc:2781] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:30.809124 21923 raft_consensus.cc:686] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [term 1 LEADER]: Becoming Leader. State: Replica: ebedae7b68614ec1910376f9aa9f6db2, State: Running, Role: LEADER
I20260808 07:06:30.809193 21923 consensus_queue.cc:237] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } }
I20260808 07:06:30.809893 21140 catalog_manager.cc:5495] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "ebedae7b68614ec1910376f9aa9f6db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebedae7b68614ec1910376f9aa9f6db2" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:30.830551 21734 raft_consensus.cc:1261] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:30.830582 21549 raft_consensus.cc:1261] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Refusing update from remote peer ebedae7b68614ec1910376f9aa9f6db2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:30.830884 21919 consensus_queue.cc:1035] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f0d6ebdc61546d593cb450476beebde" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:30.831019 21923 consensus_queue.cc:1035] T 06cd6dce9a234a8b836a9c1ebec7f210 P ebedae7b68614ec1910376f9aa9f6db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e5f409d3be45f59ceea4a2e368dceb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:06:30.835953 21907 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 07:06:32.358023 21140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:32.358139 21140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:32.359089 21140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1cbbe034cfa41c991e07c58d1746cb9: Sending DeleteTablet for 3 replicas of tablet 06cd6dce9a234a8b836a9c1ebec7f210
I20260808 07:06:32.359367 21331 tablet_service.cc:1514] Processing DeleteTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:32 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38290
I20260808 07:06:32.359368 21713 tablet_service.cc:1514] Processing DeleteTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:32 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:48696
I20260808 07:06:32.359405 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet 06cd6dce9a234a8b836a9c1ebec7f210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:32 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:54182
I20260808 07:06:32.359565 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 21240
I20260808 07:06:32.359687 22000 tablet_replica.cc:330] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: stopping tablet replica
I20260808 07:06:32.364948 22000 raft_consensus.cc:2229] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:32.365038 22000 raft_consensus.cc:2258] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:32.365361 22000 ts_tablet_manager.cc:1905] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:32.366854 21998 tablet_replica.cc:330] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: stopping tablet replica
I20260808 07:06:32.366917 21998 raft_consensus.cc:2229] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:32.367063 22000 ts_tablet_manager.cc:1918] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:06:32.367125 22000 log.cc:1196] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/06cd6dce9a234a8b836a9c1ebec7f210
I20260808 07:06:32.367354 22000 ts_tablet_manager.cc:1939] T 06cd6dce9a234a8b836a9c1ebec7f210 P 18e5f409d3be45f59ceea4a2e368dceb: Deleting consensus metadata
I20260808 07:06:32.366946 21998 raft_consensus.cc:2258] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:32.367707 21114 catalog_manager.cc:4841] TS 18e5f409d3be45f59ceea4a2e368dceb (127.18.63.66:36239): tablet 06cd6dce9a234a8b836a9c1ebec7f210 (table test-table [id=5e523b34666c471aa90e049eef022800]) successfully deleted
I20260808 07:06:32.367933 21998 ts_tablet_manager.cc:1905] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:32.369230 21998 ts_tablet_manager.cc:1918] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 07:06:32.369280 21998 log.cc:1196] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/06cd6dce9a234a8b836a9c1ebec7f210
I20260808 07:06:32.369518 21998 ts_tablet_manager.cc:1939] T 06cd6dce9a234a8b836a9c1ebec7f210 P 6f0d6ebdc61546d593cb450476beebde: Deleting consensus metadata
I20260808 07:06:32.369880 21112 catalog_manager.cc:4841] TS 6f0d6ebdc61546d593cb450476beebde (127.18.63.67:34155): tablet 06cd6dce9a234a8b836a9c1ebec7f210 (table test-table [id=5e523b34666c471aa90e049eef022800]) successfully deleted
W20260808 07:06:32.373744 21112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:33655 (error 108)
I20260808 07:06:32.374069 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 21430
I20260808 07:06:32.387564 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:32.374192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33655 (local address 127.0.0.1:46480)
0808 07:06:32.387427 (+ 13235us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:32.387508 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:33655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13197,"thread_start_us":81,"threads_started":1}
W20260808 07:06:32.387780 21112 catalog_manager.cc:4584] TS ebedae7b68614ec1910376f9aa9f6db2 (127.18.63.65:33655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06cd6dce9a234a8b836a9c1ebec7f210: Network error: Client connection negotiation failed: client connection to 127.18.63.65:33655: connect: Connection refused (error 111)
I20260808 07:06:32.393724 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 21623
I20260808 07:06:32.403611 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 21094
2026-08-08T07:06:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6069 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:32 dist-test-slave-bw88 krb5kdc[22007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:32 dist-test-slave-bw88 krb5kdc[22007](info): set up 2 sockets
Aug 08 07:06:32 dist-test-slave-bw88 krb5kdc[22007](info): commencing operation
krb5kdc: starting...
W20260808 07:06:33.587010 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:33 dist-test-slave-bw88 krb5kdc[22007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:33.711973 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38119
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:38119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:33.795542 22027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:33.795786 22027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:33.795825 22027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:33.796950 22027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:33.797024 22027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:33.797049 22027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:33.797070 22027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:33.797101 22027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:33.798607 22027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:38119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:33.798872 22027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:33.799110 22027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:33.804587 22034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:33.804529 22036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:33.804544 22027 server_base.cc:1034] running on GCE node
W20260808 07:06:33.804687 22033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:33.805024 22027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:33.805274 22027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:33.806412 22027 hybrid_clock.cc:648] HybridClock initialized: now 1786172793806403 us; error 33 us; skew 500 ppm
Aug 08 07:06:33 dist-test-slave-bw88 krb5kdc[22007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:33.808812 22027 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:33.809258 22027 webserver.cc:460] Webserver started at http://127.18.63.126:37047/ using document root <none> and password file <none>
I20260808 07:06:33.809521 22027 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:33.809571 22027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:33.809669 22027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:33.810672 22027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "e8ea539b16014f08af534b83f6db7daa"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-bw88"
server_key: "717620e368892ed80411476d70725e84"
server_key_iv: "049ebba73aa23cc13956642f1a044a81"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.810983 22027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "e8ea539b16014f08af534b83f6db7daa"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-bw88"
server_key: "717620e368892ed80411476d70725e84"
server_key_iv: "049ebba73aa23cc13956642f1a044a81"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.812300 22027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:33.812994 22047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:33.813169 22027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:33.813282 22027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "e8ea539b16014f08af534b83f6db7daa"
format_stamp: "Formatted at 2026-08-08 07:06:33 on dist-test-slave-bw88"
server_key: "717620e368892ed80411476d70725e84"
server_key_iv: "049ebba73aa23cc13956642f1a044a81"
server_key_version: "encryptionkey@0"
I20260808 07:06:33.813380 22027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:33.840804 22027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:33.842684 22027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:33.842849 22027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:33.847116 22027 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:38119
I20260808 07:06:33.847180 22112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:38119 every 8 connection(s)
I20260808 07:06:33.847532 22027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:33.848496 22114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:33.850919 22114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa: Bootstrap starting.
I20260808 07:06:33.851531 22114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:33.851835 22114 log.cc:824] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:33.852674 22114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa: No bootstrap required, opened a new log
I20260808 07:06:33.854323 22114 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } }
I20260808 07:06:33.854472 22114 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:33.854521 22114 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ea539b16014f08af534b83f6db7daa, State: Initialized, Role: FOLLOWER
I20260808 07:06:33.854668 22114 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } }
I20260808 07:06:33.854758 22114 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:33.854792 22114 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:33.854837 22114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:33.855384 22114 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } }
I20260808 07:06:33.855501 22114 leader_election.cc:302] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8ea539b16014f08af534b83f6db7daa; no voters:
I20260808 07:06:33.855685 22114 leader_election.cc:288] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:33.855715 22118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:33.855887 22118 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [term 1 LEADER]: Becoming Leader. State: Replica: e8ea539b16014f08af534b83f6db7daa, State: Running, Role: LEADER
I20260808 07:06:33.855975 22114 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:33.856041 22118 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } }
I20260808 07:06:33.856462 22120 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8ea539b16014f08af534b83f6db7daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } } }
I20260808 07:06:33.856459 22121 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8ea539b16014f08af534b83f6db7daa. Latest consensus state: current_term: 1 leader_uuid: "e8ea539b16014f08af534b83f6db7daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ea539b16014f08af534b83f6db7daa" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38119 } } }
I20260808 07:06:33.856669 22121 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:33.856866 22120 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:33.857220 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22027
I20260808 07:06:33.857342 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:33.857481 22123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:33.857579 18685 external_mini_cluster.cc:1428] Setting key 5b5c0ac942a304f22e3b6d475a5874ae
I20260808 07:06:33.858042 22123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:33.859732 22123 catalog_manager.cc:1347] Generated new cluster ID: b69646843a3147dd8e2a93fa4c6770c9
I20260808 07:06:33.859784 22123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:33 dist-test-slave-bw88 krb5kdc[22007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:33.871647 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:33.858076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.18.63.126:38119)
0808 07:06:33.858791 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:33.858799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:33.858826 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:06:33.858912 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:33.858917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:33.858966 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:33.859129 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:33.861765 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.863196 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.864369 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:33.864965 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:33.865062 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:33.867892 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:33.867897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:33.867897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:33.867909 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:33.869936 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.870338 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.870341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.870342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.870458 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:33.870686 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:33.870687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:33.870687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:33.870985 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:33.871117 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:33.871273 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:06:33.871415 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":76,"threads_started":1}
I20260808 07:06:33.874238 22123 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:33.874907 22123 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:33.882367 22123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8ea539b16014f08af534b83f6db7daa: Generated new TSK 0
I20260808 07:06:33.882541 22123 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:33.920645 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:34.006284 22151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:34.006489 22151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:34.006515 22151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:34.007596 22151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:34.007630 22151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:34.007655 22151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:34.009020 22151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:34.009208 22151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:34.009470 22151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:34.014690 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:34.014727 22162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:34.014721 22159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:34.014739 22151 server_base.cc:1034] running on GCE node
I20260808 07:06:34.015113 22151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:34.015370 22151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:34.016511 22151 hybrid_clock.cc:648] HybridClock initialized: now 1786172794016499 us; error 29 us; skew 500 ppm
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:34.019109 22151 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:34.019534 22151 webserver.cc:460] Webserver started at http://127.18.63.65:41895/ using document root <none> and password file <none>
I20260808 07:06:34.019769 22151 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:34.019838 22151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:34.019990 22151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:34.021003 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "c3b625ea44b44c12b5efbaf38f8ca0b5"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "6faa88b4b86ca0f036bfa9c31937a77a"
server_key_iv: "f7d5a3c01de48ec3ba81a1fabdc3ed59"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.021445 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "c3b625ea44b44c12b5efbaf38f8ca0b5"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "6faa88b4b86ca0f036bfa9c31937a77a"
server_key_iv: "f7d5a3c01de48ec3ba81a1fabdc3ed59"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.022804 22151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:34.023548 22174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.023718 22151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:34.023820 22151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "c3b625ea44b44c12b5efbaf38f8ca0b5"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "6faa88b4b86ca0f036bfa9c31937a77a"
server_key_iv: "f7d5a3c01de48ec3ba81a1fabdc3ed59"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.023905 22151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:34.034890 22151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:34.036716 22151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:34.036870 22151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:34.037081 22151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:34.037459 22151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:34.037524 22151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.037564 22151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:34.037591 22151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.043720 22151 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:38371
I20260808 07:06:34.043777 22321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:38371 every 8 connection(s)
I20260808 07:06:34.044076 22151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:34.046188 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22151
I20260808 07:06:34.046301 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:34.046496 18685 external_mini_cluster.cc:1428] Setting key 4580a29e92468ada1c9583e9331d8d50
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:34.055176 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.044793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46005 (local address 127.18.63.126:38119)
0808 07:06:34.044838 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:34.044841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:34.045039 (+ 198us) server_negotiation.cc:407] Connection header received
0808 07:06:34.045689 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:34.045691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:34.045710 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:34.045764 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:34.047245 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.047775 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.049370 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.049714 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.049788 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.052214 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:34.052216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:34.052216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:34.052225 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:34.053739 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.054161 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.054162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.054163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.054214 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.054474 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.054475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.054475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.054592 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:34.054670 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:34.054955 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 07:06:34.055067 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:06:34.055296 22325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.044747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38119 (local address 127.18.63.65:46005)
0808 07:06:34.044893 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:34.044938 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:34.045576 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:34.045845 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:34.045849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:34.046886 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:34.047145 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.047152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.047884 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.047886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.049252 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.049257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.049814 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.049815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.049921 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.050493 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:34.050509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:34.052118 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:34.053825 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.053832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.053834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.054058 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.054289 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.054290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.054290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.054399 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.054673 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:34.054681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:34.054818 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 07:06:34.054918 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:06:34.055076 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260808 07:06:34.055842 22323 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38119
I20260808 07:06:34.055960 22323 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:34.056146 22323 heartbeater.cc:511] Master 127.18.63.126:38119 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
I20260808 07:06:34.056697 22068 ts_manager.cc:194] Registered new tserver with Master: c3b625ea44b44c12b5efbaf38f8ca0b5 (127.18.63.65:38371)
I20260808 07:06:34.057435 22068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:46005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:34.075539 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:34.159679 22328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:34.159876 22328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:34.159899 22328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:34.161064 22328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:34.161106 22328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:34.161139 22328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:34.162535 22328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:34.162716 22328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:34.162887 22328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:34.167987 22336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:34.167986 22340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:34.168236 22328 server_base.cc:1034] running on GCE node
W20260808 07:06:34.168273 22335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:34.168514 22328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:34.168761 22328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:34.169934 22328 hybrid_clock.cc:648] HybridClock initialized: now 1786172794169919 us; error 26 us; skew 500 ppm
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:34.172356 22328 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:34.172793 22328 webserver.cc:460] Webserver started at http://127.18.63.66:39479/ using document root <none> and password file <none>
I20260808 07:06:34.173022 22328 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:34.173087 22328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:34.173239 22328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:34.174356 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "d577640aeb5a4ef880b8ee463d516993"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "aab873437dc298974d82d1619c09e69b"
server_key_iv: "1ca4ef3e6ae8012e3c2e6a5d259ab1b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.174729 22328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "d577640aeb5a4ef880b8ee463d516993"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "aab873437dc298974d82d1619c09e69b"
server_key_iv: "1ca4ef3e6ae8012e3c2e6a5d259ab1b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.176429 22328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:34.177192 22348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.177398 22328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:34.177510 22328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "d577640aeb5a4ef880b8ee463d516993"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "aab873437dc298974d82d1619c09e69b"
server_key_iv: "1ca4ef3e6ae8012e3c2e6a5d259ab1b1"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.177596 22328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:34.191041 22328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:34.192816 22328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:34.192981 22328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:34.193225 22328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:34.193599 22328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:34.193660 22328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.193696 22328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:34.193722 22328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.199594 22328 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:38457
I20260808 07:06:34.199676 22509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:38457 every 8 connection(s)
I20260808 07:06:34.199999 22328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:34.200742 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22328
I20260808 07:06:34.200847 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:34.201079 18685 external_mini_cluster.cc:1428] Setting key 8092596957e8b2bd67a8fb4bb623ccb1
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:06:34.210657 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.200875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:37135 (local address 127.18.63.126:38119)
0808 07:06:34.200924 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:34.200926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:34.201463 (+ 537us) server_negotiation.cc:407] Connection header received
0808 07:06:34.202039 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:34.202040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:34.202053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:34.202135 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:34.203046 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.203808 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.204830 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.205045 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.205155 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.207732 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:34.207734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:34.207734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:34.207742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:34.209143 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.209621 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.209622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.209622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.209688 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.209957 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.209958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.209958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.210074 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:34.210166 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:34.210474 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 07:06:34.210565 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:34.210834 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.200751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38119 (local address 127.18.63.66:37135)
0808 07:06:34.201282 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:34.201320 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:34.201936 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:34.202179 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:34.202182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:34.202758 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:34.202946 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.202951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.203916 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.203917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.204730 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.204732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.205171 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.205172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.205315 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.205972 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:34.205981 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:34.207634 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:34.209231 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.209239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.209241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.209541 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.209769 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.209770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.209770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.209882 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.210160 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:34.210167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:34.210327 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 07:06:34.210432 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 07:06:34.210591 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":95,"threads_started":1}
I20260808 07:06:34.211318 22510 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38119
I20260808 07:06:34.211406 22510 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:34.211573 22510 heartbeater.cc:511] Master 127.18.63.126:38119 requested a full tablet report, sending...
I20260808 07:06:34.211889 22068 ts_manager.cc:194] Registered new tserver with Master: d577640aeb5a4ef880b8ee463d516993 (127.18.63.66:38457)
I20260808 07:06:34.212318 22068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:37135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:34.227574 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:34.310709 22516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:34.310904 22516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:34.310926 22516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:34.312099 22516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:34.312139 22516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:34.312165 22516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:34.313633 22516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:34.313885 22516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:34.314116 22516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:34.319089 22528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:34.319202 22526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:34.319082 22525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:34.319296 22516 server_base.cc:1034] running on GCE node
I20260808 07:06:34.319626 22516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:34.319866 22516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:34.321028 22516 hybrid_clock.cc:648] HybridClock initialized: now 1786172794321023 us; error 36 us; skew 500 ppm
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:34.323567 22516 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:34.323989 22516 webserver.cc:460] Webserver started at http://127.18.63.67:42171/ using document root <none> and password file <none>
I20260808 07:06:34.324218 22516 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:34.324288 22516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:34.324441 22516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:34.325528 22516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "b9b843a0ad7141e39890917321c3c199"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "403e62108f3ee85d80321fbc7cfeb118"
server_key_iv: "ec2ffab5cdbd91dbad3df221f4b0f404"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.325903 22516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "b9b843a0ad7141e39890917321c3c199"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "403e62108f3ee85d80321fbc7cfeb118"
server_key_iv: "ec2ffab5cdbd91dbad3df221f4b0f404"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.327327 22516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:06:34.328096 22538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.328269 22516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:34.328379 22516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "b9b843a0ad7141e39890917321c3c199"
format_stamp: "Formatted at 2026-08-08 07:06:34 on dist-test-slave-bw88"
server_key: "403e62108f3ee85d80321fbc7cfeb118"
server_key_iv: "ec2ffab5cdbd91dbad3df221f4b0f404"
server_key_version: "encryptionkey@0"
I20260808 07:06:34.328451 22516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:34.339120 22516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:34.340826 22516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:34.340982 22516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:34.341224 22516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:34.341570 22516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:34.341627 22516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.341660 22516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:34.341707 22516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:34.347517 22516 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:38531
I20260808 07:06:34.347607 22695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:38531 every 8 connection(s)
I20260808 07:06:34.347883 22516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:34.352526 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22516
I20260808 07:06:34.352659 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:34.352875 18685 external_mini_cluster.cc:1428] Setting key 6a14483aa514c277aa18359656d49b32
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172794, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:34.356849 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.348621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:60197 (local address 127.18.63.126:38119)
0808 07:06:34.348684 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:34.348687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:34.348912 (+ 225us) server_negotiation.cc:407] Connection header received
0808 07:06:34.349396 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:34.349397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:34.349411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:34.349464 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:34.350353 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.350845 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.351800 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.351978 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.352029 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.354474 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:34.354475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:34.354475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:34.354481 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:34.355713 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.356070 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.356071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.356071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.356123 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:34.356298 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:34.356299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:34.356299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:34.356395 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:34.356444 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:34.356710 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:06:34.356787 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:34.357172 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:34.348597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38119 (local address 127.18.63.67:60197)
0808 07:06:34.348797 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:34.348829 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:34.349268 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:34.349525 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:34.349528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:34.350089 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:34.350272 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.350277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.350928 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.350929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.351725 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:34.351727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:34.352052 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:34.352052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:34.352149 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:34.352692 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:34.352717 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:34.354396 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:34.355781 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.355788 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.355789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.356004 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.356165 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:34.356165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:34.356165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:34.356249 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:34.356452 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:34.356457 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:34.356571 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 07:06:34.356651 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:06:34.356832 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260808 07:06:34.357661 22696 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38119
I20260808 07:06:34.357774 22696 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:34.357947 22696 heartbeater.cc:511] Master 127.18.63.126:38119 requested a full tablet report, sending...
I20260808 07:06:34.358296 22068 ts_manager.cc:194] Registered new tserver with Master: b9b843a0ad7141e39890917321c3c199 (127.18.63.67:38531)
I20260808 07:06:34.358855 22068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:60197
I20260808 07:06:34.364805 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:06:34.377652 22138 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:34.376673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.18.63.126:38119)
0808 07:06:34.376750 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:34.376752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:34.376846 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:06:34.376932 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:34.376933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:34.376943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:34.377015 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:34.377577 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34412: BlockingRecv error: recv got EOF from 127.0.0.1:34412 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:06:34.378646 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22151
I20260808 07:06:34.391589 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22328
I20260808 07:06:34.399966 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22516
I20260808 07:06:34.409776 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22027
2026-08-08T07:06:34Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2006 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22709](info): set up 2 sockets
Aug 08 07:06:34 dist-test-slave-bw88 krb5kdc[22709](info): commencing operation
krb5kdc: starting...
W20260808 07:06:35.602731 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:35.729014 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43521
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:35.815636 22723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:35.815855 22723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:35.815878 22723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:35.816970 22723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:35.817010 22723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:35.817027 22723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:35.817042 22723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:35.817055 22723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:35.818622 22723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:35.818825 22723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:35.819003 22723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:35.824227 22728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:35.824250 22731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:35.824234 22729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:35.824303 22723 server_base.cc:1034] running on GCE node
I20260808 07:06:35.824731 22723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:35.825096 22723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:35.826252 22723 hybrid_clock.cc:648] HybridClock initialized: now 1786172795826238 us; error 46 us; skew 500 ppm
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172795, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:35.832827 22723 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:35.833330 22723 webserver.cc:460] Webserver started at http://127.18.63.126:42091/ using document root <none> and password file <none>
I20260808 07:06:35.833653 22723 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:35.833721 22723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:35.833860 22723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:35.835033 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "2ff75913af4e4f78a886118b2176393d"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-bw88"
server_key: "69ad442fe837a2d61d50794773c685c2"
server_key_iv: "216b9ecc484e8ab7ae48367c84ac297d"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.835409 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "2ff75913af4e4f78a886118b2176393d"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-bw88"
server_key: "69ad442fe837a2d61d50794773c685c2"
server_key_iv: "216b9ecc484e8ab7ae48367c84ac297d"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.836767 22723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:35.837546 22740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:35.837735 22723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:35.837855 22723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "2ff75913af4e4f78a886118b2176393d"
format_stamp: "Formatted at 2026-08-08 07:06:35 on dist-test-slave-bw88"
server_key: "69ad442fe837a2d61d50794773c685c2"
server_key_iv: "216b9ecc484e8ab7ae48367c84ac297d"
server_key_version: "encryptionkey@0"
I20260808 07:06:35.837939 22723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:35.875509 22723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:35.877700 22723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:35.877873 22723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:35.882198 22723 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43521
I20260808 07:06:35.882241 22813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43521 every 8 connection(s)
I20260808 07:06:35.882844 22723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:35.883615 22814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:35.884595 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22723
I20260808 07:06:35.884708 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:35.884907 18685 external_mini_cluster.cc:1428] Setting key 43876e05c21d88fc377a536d59ecafe8
I20260808 07:06:35.886462 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d: Bootstrap starting.
I20260808 07:06:35.887291 22814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:35.887637 22814 log.cc:824] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:35.888396 22814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d: No bootstrap required, opened a new log
I20260808 07:06:35.889864 22814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } }
I20260808 07:06:35.890013 22814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:35.890038 22814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff75913af4e4f78a886118b2176393d, State: Initialized, Role: FOLLOWER
I20260808 07:06:35.890179 22814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } }
I20260808 07:06:35.890251 22814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:35.890275 22814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:35.890339 22814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:35.890959 22814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } }
I20260808 07:06:35.891103 22814 leader_election.cc:302] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ff75913af4e4f78a886118b2176393d; no voters:
I20260808 07:06:35.891314 22814 leader_election.cc:288] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:35.891345 22820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:35.891487 22820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [term 1 LEADER]: Becoming Leader. State: Replica: 2ff75913af4e4f78a886118b2176393d, State: Running, Role: LEADER
I20260808 07:06:35.891592 22814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:35.891582 22820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } }
I20260808 07:06:35.892004 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ff75913af4e4f78a886118b2176393d. Latest consensus state: current_term: 1 leader_uuid: "2ff75913af4e4f78a886118b2176393d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } } }
I20260808 07:06:35.892083 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:35.892306 22825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:35.892992 22825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:35.893512 22822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ff75913af4e4f78a886118b2176393d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ff75913af4e4f78a886118b2176393d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43521 } } }
I20260808 07:06:35.893689 22822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:35.894999 22825 catalog_manager.cc:1347] Generated new cluster ID: 5ad1717b80854aa1bb486ba4fc4e6d47
I20260808 07:06:35.895066 22825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:35.897162 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:35.885469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.18.63.126:43521)
0808 07:06:35.885660 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:35.885676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:35.885702 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:06:35.886210 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:35.886217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:35.886264 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:35.886380 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:35.887290 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.888461 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.889222 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.890337 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.890448 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:35.893513 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:35.893518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:35.893518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:35.893531 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:35.895590 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:35.895970 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:35.895973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:35.895973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:35.896069 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:35.896285 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:35.896286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:35.896287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:35.896530 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:35.896688 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:35.896799 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 07:06:35.896938 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260808 07:06:35.898108 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:35.913236 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:35.909761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.18.63.126:43521)
0808 07:06:35.909810 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:35.909812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:35.909970 (+ 158us) server_negotiation.cc:407] Connection header received
0808 07:06:35.910050 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:35.910051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:35.910062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:35.910128 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:35.910525 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.911037 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.911779 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:35.911992 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:35.912071 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:35.912771 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:35.912773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:35.912773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:35.912963 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:35.912979 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:35.912984 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:35.913054 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:35.913117 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:06:35.913141 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:35.914325 22825 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:35.914829 22825 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:35.917856 22825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ff75913af4e4f78a886118b2176393d: Generated new TSK 0
I20260808 07:06:35.918049 22825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:35.921296 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22723
2026-08-08T07:06:35Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1512 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22855](info): set up 2 sockets
Aug 08 07:06:35 dist-test-slave-bw88 krb5kdc[22855](info): commencing operation
krb5kdc: starting...
W20260808 07:06:37.294248 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:37 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:37.488888 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37501
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:37.641729 22875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:37.642146 22875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:37.642205 22875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:37.643963 22875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:37.644021 22875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:37.644083 22875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:37.644109 22875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:37.644145 22875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:37.646829 22875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:37.647271 22875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:37.647650 22875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:37.656597 22887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:37.656597 22885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:37.656611 22884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:37.656984 22875 server_base.cc:1034] running on GCE node
I20260808 07:06:37.657544 22875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:37.658195 22875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:37.659407 22875 hybrid_clock.cc:648] HybridClock initialized: now 1786172797659384 us; error 67 us; skew 500 ppm
Aug 08 07:06:37 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172797, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:37.664367 22875 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:37.665093 22875 webserver.cc:460] Webserver started at http://127.18.63.126:46079/ using document root <none> and password file <none>
I20260808 07:06:37.665795 22875 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:37.665932 22875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:37.666296 22875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:37.668386 22875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "64e9187f45ad4ec6957a4359f80fa20b"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "6066fbe0443a76e3cab3a1dc22e48ddf"
server_key_iv: "567b258bbbcab397ecc110ec1846cf58"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.668991 22875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "64e9187f45ad4ec6957a4359f80fa20b"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "6066fbe0443a76e3cab3a1dc22e48ddf"
server_key_iv: "567b258bbbcab397ecc110ec1846cf58"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.671753 22875 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:37.673270 22895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:37.673588 22875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:37.673805 22875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "64e9187f45ad4ec6957a4359f80fa20b"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "6066fbe0443a76e3cab3a1dc22e48ddf"
server_key_iv: "567b258bbbcab397ecc110ec1846cf58"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.673928 22875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:37.695462 22875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:37.699357 22875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:37.699683 22875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:37.708153 22875 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:37501
I20260808 07:06:37.708240 22961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:37501 every 8 connection(s)
I20260808 07:06:37.708971 22875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:37.711042 22962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:37.715808 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b: Bootstrap starting.
I20260808 07:06:37.716959 22962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:37.717680 22962 log.cc:824] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:37.718293 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22875
I20260808 07:06:37.718464 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:37.718796 18685 external_mini_cluster.cc:1428] Setting key 4a4cd1ca6e105cc9e0998bf608cea7f5
I20260808 07:06:37.719190 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b: No bootstrap required, opened a new log
I20260808 07:06:37.722922 22962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } }
I20260808 07:06:37.723417 22962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:37.723513 22962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e9187f45ad4ec6957a4359f80fa20b, State: Initialized, Role: FOLLOWER
I20260808 07:06:37.723826 22962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } }
I20260808 07:06:37.724054 22962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:37.724124 22962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:37.724249 22962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:37.725795 22962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } }
I20260808 07:06:37.725976 22962 leader_election.cc:302] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e9187f45ad4ec6957a4359f80fa20b; no voters:
I20260808 07:06:37.726279 22962 leader_election.cc:288] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:37.726328 22968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:37.726658 22968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [term 1 LEADER]: Becoming Leader. State: Replica: 64e9187f45ad4ec6957a4359f80fa20b, State: Running, Role: LEADER
I20260808 07:06:37.726853 22962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:37.726869 22968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } }
I20260808 07:06:37.727661 22970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e9187f45ad4ec6957a4359f80fa20b. Latest consensus state: current_term: 1 leader_uuid: "64e9187f45ad4ec6957a4359f80fa20b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } } }
I20260808 07:06:37.727818 22970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:37.728098 22969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e9187f45ad4ec6957a4359f80fa20b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e9187f45ad4ec6957a4359f80fa20b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37501 } } }
I20260808 07:06:37.728240 22969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:37.729897 22975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:37.731285 22975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:06:37 dist-test-slave-bw88 krb5kdc[22855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:37.738714 22975 catalog_manager.cc:1347] Generated new cluster ID: bdbf773496c248128316143eb0a18551
I20260808 07:06:37.738802 22975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:37.743191 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:37.719641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54060 (local address 127.18.63.126:37501)
0808 07:06:37.720022 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:37.720052 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:37.720146 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:06:37.721714 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:37.721725 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:37.721802 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:37.721956 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:37.723116 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.726430 (+ 3314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.728366 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.729917 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.730193 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:37.736140 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:37.736146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:37.736147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:37.736173 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:37.740033 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:37.740706 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:37.740712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:37.740713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:37.740915 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:37.741252 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:37.741253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:37.741253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:37.741892 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:37.742166 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:37.742313 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:06:37.742766 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":206,"threads_started":1}
I20260808 07:06:37.758407 22975 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:37.759193 22975 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:37.763794 22975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e9187f45ad4ec6957a4359f80fa20b: Generated new TSK 0
I20260808 07:06:37.763988 22975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:37.803910 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:37.935508 22995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:37.935858 22995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:37.935914 22995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:37.937644 22995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:37.937718 22995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:37.937808 22995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:37.940177 22995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:37.940580 22995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:37.940999 22995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:37.948935 23000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:37.949039 23003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:37.949000 23001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:37.949285 22995 server_base.cc:1034] running on GCE node
I20260808 07:06:37.949572 22995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:37.949911 22995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:37.951108 22995 hybrid_clock.cc:648] HybridClock initialized: now 1786172797951111 us; error 59 us; skew 500 ppm
Aug 08 07:06:37 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172797, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:37.954922 22995 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:37.955485 22995 webserver.cc:460] Webserver started at http://127.18.63.65:33085/ using document root <none> and password file <none>
I20260808 07:06:37.955783 22995 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:37.955844 22995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:37.955988 22995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:37.957415 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "f37f5df91bd3447488b4adcbf80d3639"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "943678adbb5bc6541dd06fccfac37af7"
server_key_iv: "6607940a3c42f33139fea1303a020ff9"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.957882 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "f37f5df91bd3447488b4adcbf80d3639"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "943678adbb5bc6541dd06fccfac37af7"
server_key_iv: "6607940a3c42f33139fea1303a020ff9"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.959726 22995 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:37.960652 23011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:37.960902 22995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:37.961026 22995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "f37f5df91bd3447488b4adcbf80d3639"
format_stamp: "Formatted at 2026-08-08 07:06:37 on dist-test-slave-bw88"
server_key: "943678adbb5bc6541dd06fccfac37af7"
server_key_iv: "6607940a3c42f33139fea1303a020ff9"
server_key_version: "encryptionkey@0"
I20260808 07:06:37.961114 22995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:37.977232 22995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:37.981257 22995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:37.981537 22995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:37.981894 22995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:37.982451 22995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:37.982511 22995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:37.982556 22995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:37.982580 22995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:37.993829 22995 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:45021
I20260808 07:06:37.993903 23152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:45021 every 8 connection(s)
I20260808 07:06:37.994379 22995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:06:37 dist-test-slave-bw88 krb5kdc[22855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172797, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:38.002741 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 22995
I20260808 07:06:38.002920 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:38.003276 18685 external_mini_cluster.cc:1428] Setting key be1c52879171ec7e37fa45e6d0e950dd
I20260808 07:06:38.007207 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:37.995372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:57721 (local address 127.18.63.126:37501)
0808 07:06:37.995496 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:37.995499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:37.995762 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:06:37.996399 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:37.996400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:37.996422 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:37.996481 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:37.997535 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.998153 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.999248 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.999443 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.999495 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.002223 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.002226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.002227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.002256 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.004697 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.005491 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.005492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.005493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.005599 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.006024 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.006025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.006026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.006221 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.006391 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.006873 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.007028 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 07:06:38.007526 23155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:37.995348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37501 (local address 127.18.63.65:57721)
0808 07:06:37.995637 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:37.995675 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:37.996226 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:37.996555 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:37.996558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:37.997162 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:37.997453 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:37.997460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.998281 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.998282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:37.999178 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:37.999180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:37.999512 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:37.999513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:37.999622 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.000104 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.000115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.001938 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.004885 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.004902 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.004905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.005320 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.005739 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.005741 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.005741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.005896 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.006359 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.006366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.006643 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.006828 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.007142 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":177,"threads_started":1}
I20260808 07:06:38.008543 23153 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37501
I20260808 07:06:38.008720 23153 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:38.009035 23153 heartbeater.cc:511] Master 127.18.63.126:37501 requested a full tablet report, sending...
I20260808 07:06:38.009927 22915 ts_manager.cc:194] Registered new tserver with Master: f37f5df91bd3447488b4adcbf80d3639 (127.18.63.65:45021)
I20260808 07:06:38.010923 22915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:57721
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:38.052599 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:38.194128 23159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:38.194489 23159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:38.194523 23159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:38.196372 23159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:38.196435 23159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:38.196491 23159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:38.198714 23159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:38.199050 23159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:38.199383 23159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:38.209676 23166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.209746 23169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.209676 23167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.210317 23159 server_base.cc:1034] running on GCE node
I20260808 07:06:38.210769 23159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:38.211390 23159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:38.212601 23159 hybrid_clock.cc:648] HybridClock initialized: now 1786172798212592 us; error 75 us; skew 500 ppm
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:38.218019 23159 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:38.219044 23159 webserver.cc:460] Webserver started at http://127.18.63.66:42761/ using document root <none> and password file <none>
I20260808 07:06:38.219471 23159 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:38.219592 23159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:38.219825 23159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:38.221853 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "59b003caf1c543d1b46313512cfdab74"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "781a3ba4d933a775b12783eb11c8dc0d"
server_key_iv: "9ec19edbcffdc4d52b8630961df94160"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.222404 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "59b003caf1c543d1b46313512cfdab74"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "781a3ba4d933a775b12783eb11c8dc0d"
server_key_iv: "9ec19edbcffdc4d52b8630961df94160"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.225132 23159 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260808 07:06:38.226027 23178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.226234 23159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:38.226346 23159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "59b003caf1c543d1b46313512cfdab74"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "781a3ba4d933a775b12783eb11c8dc0d"
server_key_iv: "9ec19edbcffdc4d52b8630961df94160"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.226439 23159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:38.242175 23159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:38.244233 23159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:38.244400 23159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:38.244647 23159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:38.244959 23159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:38.245010 23159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.245048 23159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:38.245091 23159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.251241 23159 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:39813
I20260808 07:06:38.251288 23322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:39813 every 8 connection(s)
I20260808 07:06:38.251617 23159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:38.252360 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23159
I20260808 07:06:38.252470 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:38.252689 18685 external_mini_cluster.cc:1428] Setting key 5230118ef3198d5f9b0da9c13be2f627
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:06:38.262036 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.252295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:38375 (local address 127.18.63.126:37501)
0808 07:06:38.252363 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.252366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.252649 (+ 283us) server_negotiation.cc:407] Connection header received
0808 07:06:38.253377 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.253378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.253391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.253441 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.254798 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.255320 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.256350 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.256558 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.256634 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.259081 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.259083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.259084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.259092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.260555 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.261011 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.261012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.261013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.261072 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.261371 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.261372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.261373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.261488 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.261559 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.261854 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.261941 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:06:38.262238 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.252286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37501 (local address 127.18.63.66:38375)
0808 07:06:38.252490 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.252543 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.253181 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.253537 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.253542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.254408 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.254704 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.254711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.255425 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.255427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.256257 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.256260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.256652 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.256653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.256787 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.257309 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.257318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.258983 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.260676 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.260684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.260686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.260920 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.261153 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.261154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.261154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.261270 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.261561 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.261569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.261709 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.261829 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.261997 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260808 07:06:38.262705 23323 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37501
I20260808 07:06:38.262818 23323 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:38.263017 23323 heartbeater.cc:511] Master 127.18.63.126:37501 requested a full tablet report, sending...
I20260808 07:06:38.263396 22915 ts_manager.cc:194] Registered new tserver with Master: 59b003caf1c543d1b46313512cfdab74 (127.18.63.66:39813)
I20260808 07:06:38.263916 22915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:38375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:38.277940 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:38.361021 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:38.361239 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:38.361263 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:38.362457 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:38.362501 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:38.362526 23329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:38.363994 23329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:38.364234 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:38.364495 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:38.369709 23339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.369721 23337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:38.369735 23336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:38.370002 23329 server_base.cc:1034] running on GCE node
I20260808 07:06:38.370203 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:38.370457 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:38.371618 23329 hybrid_clock.cc:648] HybridClock initialized: now 1786172798371601 us; error 27 us; skew 500 ppm
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:38.374246 23329 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:38.374670 23329 webserver.cc:460] Webserver started at http://127.18.63.67:46763/ using document root <none> and password file <none>
I20260808 07:06:38.374919 23329 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:38.374984 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:38.375128 23329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:38.376164 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "8eee443184e8478b9ef8eb21b52396d1"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "6f7af8687ae0021dce8bb6ef6d4525c6"
server_key_iv: "f25bc691aaad7e241ed8aadd21b76fb4"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.376529 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "8eee443184e8478b9ef8eb21b52396d1"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "6f7af8687ae0021dce8bb6ef6d4525c6"
server_key_iv: "f25bc691aaad7e241ed8aadd21b76fb4"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.377955 23329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:06:38.378652 23350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.378820 23329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:38.378909 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "8eee443184e8478b9ef8eb21b52396d1"
format_stamp: "Formatted at 2026-08-08 07:06:38 on dist-test-slave-bw88"
server_key: "6f7af8687ae0021dce8bb6ef6d4525c6"
server_key_iv: "f25bc691aaad7e241ed8aadd21b76fb4"
server_key_version: "encryptionkey@0"
I20260808 07:06:38.378985 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:38.385960 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:38.388149 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:38.388301 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:38.388525 23329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:38.388818 23329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:38.388868 23329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.388909 23329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:38.388938 23329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:38.394836 23329 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:42037
I20260808 07:06:38.394912 23495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:42037 every 8 connection(s)
I20260808 07:06:38.395213 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:38.402837 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23329
I20260808 07:06:38.402946 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:38.403182 18685 external_mini_cluster.cc:1428] Setting key 4550d24250ca2837e4a19cc5476f0fec
I20260808 07:06:38.404150 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.395934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:38689 (local address 127.18.63.126:37501)
0808 07:06:38.396093 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.396095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.396211 (+ 116us) server_negotiation.cc:407] Connection header received
0808 07:06:38.396678 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.396679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.396690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.396743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.397706 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.398220 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.399157 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.399340 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.399390 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.401450 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.401451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.401451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.401458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.402646 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.403155 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.403156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.403156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.403209 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.403495 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.403497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.403497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.403621 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.403700 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.403969 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.404052 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 07:06:38.404340 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.395891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37501 (local address 127.18.63.67:38689)
0808 07:06:38.396081 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:38.396116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:38.396595 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:38.396794 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:38.396797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:38.397409 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:38.397611 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.397616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.398299 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.398299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.399085 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:38.399087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.399409 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.399409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:38.399513 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.400005 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:38.400013 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:38.401374 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:38.402720 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.402728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.402731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.403052 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.403286 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:38.403287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:38.403288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:38.403434 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:38.403691 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:38.403699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:38.403839 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:06:38.403947 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:06:38.404120 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260808 07:06:38.404752 23496 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37501
I20260808 07:06:38.404857 23496 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:38.405030 23496 heartbeater.cc:511] Master 127.18.63.126:37501 requested a full tablet report, sending...
I20260808 07:06:38.405411 22915 ts_manager.cc:194] Registered new tserver with Master: 8eee443184e8478b9ef8eb21b52396d1 (127.18.63.67:42037)
I20260808 07:06:38.405784 22915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:38689
I20260808 07:06:38.414808 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[22855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172798, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:38.440362 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:38.432459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.18.63.126:37501)
0808 07:06:38.432556 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:38.432559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:38.432668 (+ 109us) server_negotiation.cc:407] Connection header received
0808 07:06:38.432766 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:38.432767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:38.432778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:38.432849 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:38.433480 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.433994 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.435010 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:38.435236 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:38.435314 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:38.437899 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:38.437901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:38.437901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:38.437909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:38.439147 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.439513 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.439514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.439515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.439573 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:38.439776 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:38.439776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:38.439777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:38.439937 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:38.440059 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:38.440158 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 07:06:38.440248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 07:06:38.440634 22915 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54062
I20260808 07:06:38.441100 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22995
I20260808 07:06:38.454288 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23159
I20260808 07:06:38.461778 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23329
I20260808 07:06:38.477756 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 22875
2026-08-08T07:06:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2556 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[23509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[23509](info): set up 2 sockets
Aug 08 07:06:38 dist-test-slave-bw88 krb5kdc[23509](info): commencing operation
krb5kdc: starting...
W20260808 07:06:39.654896 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:39 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:39.779692 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33477
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:39.867233 23529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:39.867436 23529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:39.867457 23529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:39.868594 23529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:39.868649 23529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:39.868664 23529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:39.868677 23529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:39.868729 23529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:39.870311 23529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:39.870607 23529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:39.870853 23529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:39.876123 23529 server_base.cc:1034] running on GCE node
W20260808 07:06:39.876101 23542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:39.876101 23537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:39.876094 23539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:39.876461 23529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:39.876823 23529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:39.877990 23529 hybrid_clock.cc:648] HybridClock initialized: now 1786172799877974 us; error 28 us; skew 500 ppm
Aug 08 07:06:39 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172799, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:39.880484 23529 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:39.880990 23529 webserver.cc:460] Webserver started at http://127.18.63.126:38239/ using document root <none> and password file <none>
I20260808 07:06:39.881196 23529 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:39.881279 23529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:39.881431 23529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:39.882472 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "9409d6b3c53c45aaab03db2c17e35ece"
format_stamp: "Formatted at 2026-08-08 07:06:39 on dist-test-slave-bw88"
server_key: "ba52d4033de3eb660fb4f509cd7817e3"
server_key_iv: "f5c728119012a9f5563ca78e4dcc256c"
server_key_version: "encryptionkey@0"
I20260808 07:06:39.882788 23529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "9409d6b3c53c45aaab03db2c17e35ece"
format_stamp: "Formatted at 2026-08-08 07:06:39 on dist-test-slave-bw88"
server_key: "ba52d4033de3eb660fb4f509cd7817e3"
server_key_iv: "f5c728119012a9f5563ca78e4dcc256c"
server_key_version: "encryptionkey@0"
I20260808 07:06:39.884090 23529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:39.884768 23554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:39.884924 23529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:39.885011 23529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "9409d6b3c53c45aaab03db2c17e35ece"
format_stamp: "Formatted at 2026-08-08 07:06:39 on dist-test-slave-bw88"
server_key: "ba52d4033de3eb660fb4f509cd7817e3"
server_key_iv: "f5c728119012a9f5563ca78e4dcc256c"
server_key_version: "encryptionkey@0"
I20260808 07:06:39.885066 23529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:39.894333 23529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:39.896500 23529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:39.896617 23529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:39.900511 23529 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33477
I20260808 07:06:39.900569 23618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33477 every 8 connection(s)
I20260808 07:06:39.901052 23529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:39.901822 23620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:39.904135 23620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece: Bootstrap starting.
I20260808 07:06:39.904668 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23529
I20260808 07:06:39.904793 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:39.904798 23620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:39.904981 18685 external_mini_cluster.cc:1428] Setting key 9078fe2917c9c14c259edf23e7523dc9
I20260808 07:06:39.905145 23620 log.cc:824] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:39.906026 23620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece: No bootstrap required, opened a new log
I20260808 07:06:39.908385 23620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } }
I20260808 07:06:39.908557 23620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:39.908592 23620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9409d6b3c53c45aaab03db2c17e35ece, State: Initialized, Role: FOLLOWER
I20260808 07:06:39.908711 23620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } }
I20260808 07:06:39.908782 23620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:39.908810 23620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:39.908852 23620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:39.909562 23620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } }
I20260808 07:06:39.909686 23620 leader_election.cc:302] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9409d6b3c53c45aaab03db2c17e35ece; no voters:
I20260808 07:06:39.909864 23620 leader_election.cc:288] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:39.909904 23626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:39.910122 23626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [term 1 LEADER]: Becoming Leader. State: Replica: 9409d6b3c53c45aaab03db2c17e35ece, State: Running, Role: LEADER
I20260808 07:06:39.910130 23620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:39.910323 23626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } }
I20260808 07:06:39.910724 23626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9409d6b3c53c45aaab03db2c17e35ece. Latest consensus state: current_term: 1 leader_uuid: "9409d6b3c53c45aaab03db2c17e35ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } } }
I20260808 07:06:39.910991 23626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:39.910913 23628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9409d6b3c53c45aaab03db2c17e35ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9409d6b3c53c45aaab03db2c17e35ece" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33477 } } }
I20260808 07:06:39.911496 23628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:39.911633 23636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:06:39 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:39.912127 23636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:39.913831 23636 catalog_manager.cc:1347] Generated new cluster ID: c04651f40f0f401f89edc3cd520944a0
I20260808 07:06:39.913890 23636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:39.916388 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:39.905522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.18.63.126:33477)
0808 07:06:39.905707 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:39.905717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:39.905757 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:06:39.906234 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:39.906241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:39.906289 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:39.906408 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:39.907198 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.908070 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.909122 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:39.909604 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:39.909722 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:39.912659 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:39.912663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:39.912664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:39.912675 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:39.914727 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:39.915124 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:39.915127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:39.915128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:39.915246 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:39.915463 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:39.915464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:39.915465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:39.915780 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:39.915945 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:39.916032 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:06:39.916176 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260808 07:06:39.923745 23636 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:39.924458 23636 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:39.930073 23636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9409d6b3c53c45aaab03db2c17e35ece: Generated new TSK 0
I20260808 07:06:39.930238 23636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:39.960714 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:40.046977 23650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:40.047186 23650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:40.047210 23650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:40.048367 23650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:40.048408 23650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:40.048460 23650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:40.049993 23650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:40.050220 23650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:40.050495 23650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:40.055789 23658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.055794 23659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.055794 23661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:40.056059 23650 server_base.cc:1034] running on GCE node
I20260808 07:06:40.056300 23650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:40.056530 23650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:40.057688 23650 hybrid_clock.cc:648] HybridClock initialized: now 1786172800057685 us; error 40 us; skew 500 ppm
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:40.060283 23650 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:40.060750 23650 webserver.cc:460] Webserver started at http://127.18.63.65:44369/ using document root <none> and password file <none>
I20260808 07:06:40.060945 23650 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:40.060986 23650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:40.061092 23650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:40.062186 23650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "9f2bf0cd45004c92a3af152b08dec532"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "894a416000d814b0e616f345d05ff02e"
server_key_iv: "3e4873c2dffd2102b21820ab36bea6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.062538 23650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "9f2bf0cd45004c92a3af152b08dec532"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "894a416000d814b0e616f345d05ff02e"
server_key_iv: "3e4873c2dffd2102b21820ab36bea6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.063796 23650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:40.064491 23670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.064663 23650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:40.064749 23650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "9f2bf0cd45004c92a3af152b08dec532"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "894a416000d814b0e616f345d05ff02e"
server_key_iv: "3e4873c2dffd2102b21820ab36bea6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.064805 23650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:40.085598 23650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:40.087581 23650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:40.087702 23650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:40.087903 23650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:40.088246 23650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:40.088289 23650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.088330 23650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:40.088354 23650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.094390 23650 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:33497
I20260808 07:06:40.094475 23817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:33497 every 8 connection(s)
I20260808 07:06:40.094790 23650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:40.096158 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23650
I20260808 07:06:40.096249 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:40.096457 18685 external_mini_cluster.cc:1428] Setting key a3606b4a2af23e9acc3cd96ffa75da04
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:40.106426 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.095577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:60991 (local address 127.18.63.126:33477)
0808 07:06:40.095637 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:40.095639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:40.095865 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:06:40.097020 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:40.097021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:40.097032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:40.097110 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:40.097854 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.098620 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.100089 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.100302 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.100402 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.103137 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:40.103138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:40.103139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:40.103147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:40.104738 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.105248 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.105249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.105249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.105308 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.105701 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.105702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.105702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.105825 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:40.105887 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:40.106252 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 07:06:40.106349 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:40.106647 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.095566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33477 (local address 127.18.63.65:60991)
0808 07:06:40.095736 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:40.095774 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:40.096929 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:40.097211 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:40.097217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:40.097551 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:40.097757 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.097763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.098716 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.098717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.099965 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.099969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.100413 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.100414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.100557 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.101130 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:40.101139 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:40.103008 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:40.104831 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.104839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.104842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.105157 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.105471 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.105472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.105472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.105624 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.105899 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:40.105906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:40.106085 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 07:06:40.106221 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:06:40.106401 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
I20260808 07:06:40.107432 23818 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33477
I20260808 07:06:40.107522 23818 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:40.107740 23818 heartbeater.cc:511] Master 127.18.63.126:33477 requested a full tablet report, sending...
I20260808 07:06:40.108309 23578 ts_manager.cc:194] Registered new tserver with Master: 9f2bf0cd45004c92a3af152b08dec532 (127.18.63.65:33497)
I20260808 07:06:40.108968 23578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:60991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:40.124477 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:40.210193 23825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:40.210462 23825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:40.210492 23825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:40.211781 23825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:40.211826 23825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:40.211854 23825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:40.213370 23825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:40.213567 23825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:40.213748 23825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:40.219206 23840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.219213 23835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.219202 23836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:40.219674 23825 server_base.cc:1034] running on GCE node
I20260808 07:06:40.219935 23825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:40.220182 23825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:40.221367 23825 hybrid_clock.cc:648] HybridClock initialized: now 1786172800221359 us; error 37 us; skew 500 ppm
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:40.223863 23825 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:40.224296 23825 webserver.cc:460] Webserver started at http://127.18.63.66:42609/ using document root <none> and password file <none>
I20260808 07:06:40.224545 23825 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:40.224613 23825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:40.224756 23825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:40.225844 23825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "dace1505989743b7ad4ccc10132a8b98"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "d1e329a5f5f51da3ce0058ecd3cf67dc"
server_key_iv: "4cf15a6e5a38962311cc66a22f74e2ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.226222 23825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "dace1505989743b7ad4ccc10132a8b98"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "d1e329a5f5f51da3ce0058ecd3cf67dc"
server_key_iv: "4cf15a6e5a38962311cc66a22f74e2ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.227532 23825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:40.228250 23848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.228427 23825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:40.228531 23825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "dace1505989743b7ad4ccc10132a8b98"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "d1e329a5f5f51da3ce0058ecd3cf67dc"
server_key_iv: "4cf15a6e5a38962311cc66a22f74e2ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.228600 23825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:40.251871 23825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:40.254452 23825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:40.254629 23825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:40.254889 23825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:40.255218 23825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:40.255271 23825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.255311 23825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:40.255343 23825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.261579 23825 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:35927
I20260808 07:06:40.261641 23990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:35927 every 8 connection(s)
I20260808 07:06:40.261967 23825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:40.270243 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23825
I20260808 07:06:40.270370 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:40.270581 18685 external_mini_cluster.cc:1428] Setting key fbc9038fdfdf3789e42a72c6f9e54df6
I20260808 07:06:40.271620 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.262742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:48025 (local address 127.18.63.126:33477)
0808 07:06:40.262810 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:40.262813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:40.263048 (+ 235us) server_negotiation.cc:407] Connection header received
0808 07:06:40.264019 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:40.264020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:40.264029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:40.264078 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:40.264693 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.265191 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.266348 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.266538 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.266594 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.268808 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:40.268809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:40.268810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:40.268817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:40.270133 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.270604 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.270606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.270606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.270695 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.270961 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.270962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.270962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.271084 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:40.271152 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:40.271441 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:06:40.271529 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:06:40.271809 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.262729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33477 (local address 127.18.63.66:48025)
0808 07:06:40.262917 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:40.262956 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:40.263921 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:40.264144 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:40.264147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:40.264407 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:40.264621 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.264626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.265422 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.265423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.266273 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.266275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.266613 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.266614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.266722 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.267194 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:40.267203 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:40.268706 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:40.270215 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.270224 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.270227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.270529 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.270759 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.270760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.270760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.270883 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.271163 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:40.271170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:40.271307 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 07:06:40.271406 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 07:06:40.271567 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260808 07:06:40.272317 23991 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33477
I20260808 07:06:40.272447 23991 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:40.272663 23991 heartbeater.cc:511] Master 127.18.63.126:33477 requested a full tablet report, sending...
I20260808 07:06:40.273095 23578 ts_manager.cc:194] Registered new tserver with Master: dace1505989743b7ad4ccc10132a8b98 (127.18.63.66:35927)
I20260808 07:06:40.273590 23578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:48025
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:40.298265 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:40.385063 23998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:40.385272 23998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:40.385295 23998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:40.386482 23998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:40.386528 23998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:40.386583 23998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:40.388347 23998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:40.388589 23998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:40.388826 23998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:40.394420 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.394415 24006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:40.394411 24005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:40.394840 23998 server_base.cc:1034] running on GCE node
I20260808 07:06:40.395072 23998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:40.395306 23998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:40.396467 23998 hybrid_clock.cc:648] HybridClock initialized: now 1786172800396452 us; error 28 us; skew 500 ppm
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:40.399168 23998 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:40.399751 23998 webserver.cc:460] Webserver started at http://127.18.63.67:36319/ using document root <none> and password file <none>
I20260808 07:06:40.400027 23998 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:40.400101 23998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:40.400254 23998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:40.401465 23998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "14c3b4b2061549cba69d8a570f44573f"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "08ab9daadb444021b97f7f0398d7e856"
server_key_iv: "1ecd96d55c59d28ef9218f4feb7b33db"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.401867 23998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "14c3b4b2061549cba69d8a570f44573f"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "08ab9daadb444021b97f7f0398d7e856"
server_key_iv: "1ecd96d55c59d28ef9218f4feb7b33db"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.403326 23998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:40.404155 24017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.404333 23998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:40.404443 23998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "14c3b4b2061549cba69d8a570f44573f"
format_stamp: "Formatted at 2026-08-08 07:06:40 on dist-test-slave-bw88"
server_key: "08ab9daadb444021b97f7f0398d7e856"
server_key_iv: "1ecd96d55c59d28ef9218f4feb7b33db"
server_key_version: "encryptionkey@0"
I20260808 07:06:40.404521 23998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:40.427059 23998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:40.429065 23998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:40.429239 23998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:40.429548 23998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:40.429919 23998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:40.429975 23998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.430012 23998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:40.430042 23998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:40.436401 23998 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:37375
I20260808 07:06:40.436458 24178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:37375 every 8 connection(s)
I20260808 07:06:40.436784 23998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:40.444253 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 23998
I20260808 07:06:40.444360 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:40.444664 18685 external_mini_cluster.cc:1428] Setting key 2281b780f16e6a0b93555529b2fdc27c
I20260808 07:06:40.446709 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.437577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:54077 (local address 127.18.63.126:33477)
0808 07:06:40.437640 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:40.437643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:40.437867 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:06:40.438781 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:40.438782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:40.438793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:40.438846 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:40.439432 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.439942 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.440992 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.441187 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.441237 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.443596 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:40.443597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:40.443597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:40.443605 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:40.445246 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.445837 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.445838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.445838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.445891 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.446134 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.446135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.446135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.446237 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:40.446287 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:40.446575 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 07:06:40.446646 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:40.446904 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.437582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33477 (local address 127.18.63.67:54077)
0808 07:06:40.437742 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:40.437779 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:40.438691 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:40.438904 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:40.438907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:40.439167 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:40.439364 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.439368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.440030 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.440030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.440916 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:40.440918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.441261 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.441262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:40.441409 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.441907 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:40.441916 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:40.443491 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:40.445326 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.445354 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.445356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.445753 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.445955 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:40.445956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:40.445956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:40.446073 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:40.446293 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:40.446297 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:40.446459 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 07:06:40.446555 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:06:40.446689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260808 07:06:40.447427 24181 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33477
I20260808 07:06:40.447518 24181 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:40.447679 24181 heartbeater.cc:511] Master 127.18.63.126:33477 requested a full tablet report, sending...
I20260808 07:06:40.448035 23578 ts_manager.cc:194] Registered new tserver with Master: 14c3b4b2061549cba69d8a570f44573f (127.18.63.67:37375)
I20260808 07:06:40.448405 23578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:54077
I20260808 07:06:40.456430 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:40.480902 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.473437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.18.63.65:33497)
0808 07:06:40.473605 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:40.473610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:40.473621 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:40.473650 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:40.473652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:40.473666 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:40.473739 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:40.474367 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.475094 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.475812 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.476159 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.476215 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.478429 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:40.478438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:40.478439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:40.478446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:40.479839 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.480215 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.480218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.480218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.480275 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.480443 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.480444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.480444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.480577 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:40.480681 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:40.480752 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:06:40.480816 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[23509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:40.488826 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:40.482280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.18.63.126:33477)
0808 07:06:40.482327 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:40.482328 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:40.482337 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:40.482363 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:40.482364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:40.482374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:40.482418 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:40.482865 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.483394 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.484141 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:40.484333 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:40.484399 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:40.486638 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:40.486640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:40.486640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:40.486647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:40.487818 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.488138 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.488138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.488139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.488186 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:40.488340 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:40.488340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:40.488340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:40.488505 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:40.488588 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:40.488694 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 07:06:40.488759 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:06:40.489050 23578 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41758
I20260808 07:06:40.489408 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23650
I20260808 07:06:40.504117 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23825
I20260808 07:06:40.517724 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23998
I20260808 07:06:40.527196 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 23529
2026-08-08T07:06:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2049 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[24198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[24198](info): set up 2 sockets
Aug 08 07:06:40 dist-test-slave-bw88 krb5kdc[24198](info): commencing operation
krb5kdc: starting...
W20260808 07:06:41.723552 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:41 dist-test-slave-bw88 krb5kdc[24198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:41.846146 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46151
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:46151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:41.931591 24215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:41.931845 24215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:41.931890 24215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:41.933064 24215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:41.933142 24215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:41.933169 24215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:41.933203 24215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:41.933233 24215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:41.934777 24215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:46151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:41.935060 24215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:41.935303 24215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:06:41.940383 24215 server_base.cc:1034] running on GCE node
W20260808 07:06:41.940351 24220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.940340 24226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:41.940326 24221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:41.940857 24215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:41.941219 24215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:41.942381 24215 hybrid_clock.cc:648] HybridClock initialized: now 1786172801942357 us; error 39 us; skew 500 ppm
Aug 08 07:06:41 dist-test-slave-bw88 krb5kdc[24198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:41.949517 24215 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:41.949905 24215 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:41.949956 24215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:41.950109 24215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:41.951179 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "561b81fdb6de49cab91b7fbc41dcbea0"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-bw88"
server_key: "dd8aaaf6ad003ec285df9e1531e0c040"
server_key_iv: "2c127b3fe026d459ded74c965b51db96"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.951550 24215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "561b81fdb6de49cab91b7fbc41dcbea0"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-bw88"
server_key: "dd8aaaf6ad003ec285df9e1531e0c040"
server_key_iv: "2c127b3fe026d459ded74c965b51db96"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.952937 24215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:41.953742 24234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:41.953903 24215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:41.954011 24215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "561b81fdb6de49cab91b7fbc41dcbea0"
format_stamp: "Formatted at 2026-08-08 07:06:41 on dist-test-slave-bw88"
server_key: "dd8aaaf6ad003ec285df9e1531e0c040"
server_key_iv: "2c127b3fe026d459ded74c965b51db96"
server_key_version: "encryptionkey@0"
I20260808 07:06:41.954082 24215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:41.966161 24215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:41.967936 24215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:41.968088 24215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:41.972415 24215 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:46151
I20260808 07:06:41.972442 24310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:46151 every 8 connection(s)
I20260808 07:06:41.972730 24215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:41.973930 24311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:41.976421 24311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: Bootstrap starting.
I20260808 07:06:41.977051 24311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:41.977315 24311 log.cc:824] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:41.978075 24311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: No bootstrap required, opened a new log
I20260808 07:06:41.979671 24311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } }
I20260808 07:06:41.979809 24311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:41.979863 24311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561b81fdb6de49cab91b7fbc41dcbea0, State: Initialized, Role: FOLLOWER
I20260808 07:06:41.980010 24311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } }
I20260808 07:06:41.980070 24311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:41.980125 24311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:41.980175 24311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:41.981098 24311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } }
I20260808 07:06:41.981132 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 24215
I20260808 07:06:41.981241 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:41.981247 24311 leader_election.cc:302] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 561b81fdb6de49cab91b7fbc41dcbea0; no voters:
I20260808 07:06:41.981454 24311 leader_election.cc:288] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:41.981490 18685 external_mini_cluster.cc:1428] Setting key f7a080dc872a14e8aff5b43f1bcaea6a
I20260808 07:06:41.981496 24317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:41.981707 24317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [term 1 LEADER]: Becoming Leader. State: Replica: 561b81fdb6de49cab91b7fbc41dcbea0, State: Running, Role: LEADER
I20260808 07:06:41.981781 24311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:41.981823 24317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } }
I20260808 07:06:41.982302 24318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } } }
I20260808 07:06:41.982386 24318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:41.982560 24319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 561b81fdb6de49cab91b7fbc41dcbea0. Latest consensus state: current_term: 1 leader_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561b81fdb6de49cab91b7fbc41dcbea0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46151 } } }
I20260808 07:06:41.982628 24319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:41.982960 24336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:41.983601 24336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:41.985301 24336 catalog_manager.cc:1347] Generated new cluster ID: d13f3b5bb55a44a8bba44aae946fa2a9
I20260808 07:06:41.985384 24336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:41 dist-test-slave-bw88 krb5kdc[24198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:41.992832 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:41.982131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.18.63.126:46151)
0808 07:06:41.982334 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:41.982344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:41.982385 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:06:41.983083 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:41.983092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:41.983157 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:41.983322 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:41.984093 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.985426 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.986191 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:41.986630 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:41.986744 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:41.988999 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:41.989003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:41.989004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:41.989016 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:41.991263 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.991661 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.991665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.991665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.991756 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:41.991969 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:41.991969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:41.991970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:41.992208 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:41.992389 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:41.992459 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:06:41.992607 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260808 07:06:41.998467 24336 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:41.999325 24336 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:42.001969 24336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: Generated new TSK 0
I20260808 07:06:42.002137 24336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:42.036408 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:42.120554 24350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:42.120779 24350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:42.120820 24350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:42.122048 24350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:42.122108 24350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:42.122162 24350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:42.123664 24350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:42.123919 24350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:42.124151 24350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:42.129254 24363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.129307 24360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.129307 24361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.129439 24350 server_base.cc:1034] running on GCE node
I20260808 07:06:42.129757 24350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:42.129987 24350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:42.131150 24350 hybrid_clock.cc:648] HybridClock initialized: now 1786172802131137 us; error 30 us; skew 500 ppm
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:42.133801 24350 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:42.134140 24350 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:42.134208 24350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:42.134357 24350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:42.135391 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "ce22c2be60db41ae877f750a067d18df"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "8c7e0e9a49256d53d218c128ea8052ff"
server_key_iv: "41b7ebaba93f58c687a69fb6c8701cca"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.135767 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "ce22c2be60db41ae877f750a067d18df"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "8c7e0e9a49256d53d218c128ea8052ff"
server_key_iv: "41b7ebaba93f58c687a69fb6c8701cca"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.137051 24350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:42.137832 24369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.138005 24350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.138108 24350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "ce22c2be60db41ae877f750a067d18df"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "8c7e0e9a49256d53d218c128ea8052ff"
server_key_iv: "41b7ebaba93f58c687a69fb6c8701cca"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.138180 24350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:42.146029 24350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:42.148229 24350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:42.148381 24350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:42.148605 24350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:42.148909 24350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:42.148962 24350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.149000 24350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:42.149029 24350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.155459 24350 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:43197
I20260808 07:06:42.155499 24517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:43197 every 8 connection(s)
I20260808 07:06:42.155766 24350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:42.161968 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 24350
I20260808 07:06:42.162060 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:42.162312 18685 external_mini_cluster.cc:1428] Setting key a65424b0630f4779f832eb02c0aa78d5
I20260808 07:06:42.166384 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.156418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48251 (local address 127.18.63.126:46151)
0808 07:06:42.156471 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.156473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.156787 (+ 314us) server_negotiation.cc:407] Connection header received
0808 07:06:42.157256 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.157257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.157271 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.157324 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.158487 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.158996 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.159992 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.160286 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.160361 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.162787 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.162788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.162788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:42.162795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.164673 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.165256 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.165257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.165258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.165314 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.165717 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.165718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.165719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.165827 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.165882 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.166238 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.166320 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:42.166708 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.156436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46151 (local address 127.18.63.65:48251)
0808 07:06:42.156680 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.156714 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.157175 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.157473 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.157476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.157805 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:42.158306 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.158312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.159104 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.159105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.159917 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.159919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.160372 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.160372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.160494 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.160984 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:42.160993 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:42.162708 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:42.164766 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.164797 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.164799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.165167 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.165469 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.165470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.165471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.165641 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.165897 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:42.165904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:42.166084 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.166224 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.166426 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260808 07:06:42.167277 24518 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46151
I20260808 07:06:42.167367 24518 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:42.167534 24518 heartbeater.cc:511] Master 127.18.63.126:46151 requested a full tablet report, sending...
I20260808 07:06:42.168147 24262 ts_manager.cc:194] Registered new tserver with Master: ce22c2be60db41ae877f750a067d18df (127.18.63.65:43197)
I20260808 07:06:42.169145 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:48251
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:42.190145 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:42.285234 24525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:42.285514 24525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:42.285559 24525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:42.286741 24525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:42.286792 24525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:42.286835 24525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:42.288853 24525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:42.289139 24525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:42.289469 24525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:42.295589 24535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.295753 24525 server_base.cc:1034] running on GCE node
W20260808 07:06:42.295568 24537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.295795 24532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.296111 24525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:42.296375 24525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:42.297546 24525 hybrid_clock.cc:648] HybridClock initialized: now 1786172802297517 us; error 50 us; skew 500 ppm
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:42.300551 24525 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:42.301007 24525 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:42.301088 24525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:42.301275 24525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:42.302951 24525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "7dd45936eefc4f40ab43e1c1b99e7abb"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "d3eb1c981d52f9ebc17db9877fd42a44"
server_key_iv: "e589625dd5db819088af74344fecfa58"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.303457 24525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "7dd45936eefc4f40ab43e1c1b99e7abb"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "d3eb1c981d52f9ebc17db9877fd42a44"
server_key_iv: "e589625dd5db819088af74344fecfa58"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.305555 24525 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:06:42.306555 24544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.306777 24525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:42.306880 24525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "7dd45936eefc4f40ab43e1c1b99e7abb"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "d3eb1c981d52f9ebc17db9877fd42a44"
server_key_iv: "e589625dd5db819088af74344fecfa58"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.306963 24525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:42.324049 24525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:42.327610 24525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:42.327790 24525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:42.328063 24525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:42.328447 24525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:42.328500 24525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.328549 24525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:42.328577 24525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.334990 24525 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:40441
I20260808 07:06:42.335085 24699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:40441 every 8 connection(s)
I20260808 07:06:42.335332 24525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:42.336560 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 24525
I20260808 07:06:42.336684 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:42.336948 18685 external_mini_cluster.cc:1428] Setting key f9c136b23778d3c1eb5793ad55fe006e
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:42.347795 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.337654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:58997 (local address 127.18.63.126:46151)
0808 07:06:42.337695 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.337697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.337706 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:42.338286 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.338287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.338300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.338354 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.339306 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.339939 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.341583 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.341800 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.341889 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.344316 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.344318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.344318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:42.344326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.346068 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.346612 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.346613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.346613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.346683 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.346982 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.346983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.346983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.347138 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.347220 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.347583 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.347685 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:42.347963 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.337038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46151 (local address 127.18.63.66:58997)
0808 07:06:42.337497 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.337540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.338173 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.338424 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.338428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.338701 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:42.339210 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.339215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.340049 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.340050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.341503 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.341508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.341939 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.341940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.342112 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.342703 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:42.342713 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:42.344210 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:42.346176 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.346185 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.346188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.346511 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.346758 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.346759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.346760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.346904 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.347224 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:42.347231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:42.347409 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.347546 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.347732 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":382,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:06:42.348574 24700 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46151
I20260808 07:06:42.348695 24700 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:42.348857 24700 heartbeater.cc:511] Master 127.18.63.126:46151 requested a full tablet report, sending...
I20260808 07:06:42.349303 24262 ts_manager.cc:194] Registered new tserver with Master: 7dd45936eefc4f40ab43e1c1b99e7abb (127.18.63.66:40441)
I20260808 07:06:42.349689 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:58997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:42.369966 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:42.453529 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:42.453751 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:42.453794 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:42.455036 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:42.455092 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:42.455135 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:42.456712 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:42.456979 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:42.457235 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:42.462333 24715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.462568 24706 server_base.cc:1034] running on GCE node
W20260808 07:06:42.462337 24719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.462337 24713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.462924 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:42.463167 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:42.464330 24706 hybrid_clock.cc:648] HybridClock initialized: now 1786172802464307 us; error 43 us; skew 500 ppm
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:42.467054 24706 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:42.467484 24706 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:42.467556 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:42.467693 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:42.468753 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "3645c789851545038dfbc07be4414cc3"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "422ad3aa8960673a45fb0574e0b80e69"
server_key_iv: "516a6e75621a025b0355569c625dc925"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.469214 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "3645c789851545038dfbc07be4414cc3"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "422ad3aa8960673a45fb0574e0b80e69"
server_key_iv: "516a6e75621a025b0355569c625dc925"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.470649 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:42.471472 24725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.471729 24706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:42.471827 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "3645c789851545038dfbc07be4414cc3"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "422ad3aa8960673a45fb0574e0b80e69"
server_key_iv: "516a6e75621a025b0355569c625dc925"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.471887 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:42.490832 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:42.492921 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:42.493099 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:42.493414 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:42.493781 24706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:42.493818 24706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.493860 24706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:42.493887 24706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.499850 24706 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:43321
I20260808 07:06:42.499923 24876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:43321 every 8 connection(s)
I20260808 07:06:42.500195 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:42.505467 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 24706
I20260808 07:06:42.505602 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:42.505780 18685 external_mini_cluster.cc:1428] Setting key 6800f980a34a4d106fd12f5eca922443
Aug 08 07:06:42 dist-test-slave-bw88 krb5kdc[24198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172802, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:42.509370 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.500914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:46743 (local address 127.18.63.126:46151)
0808 07:06:42.500972 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.500974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.501217 (+ 243us) server_negotiation.cc:407] Connection header received
0808 07:06:42.501837 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.501839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.501857 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.501919 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.502926 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.503435 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.504474 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.504677 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.504762 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.507019 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.507020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.507020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:42.507027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.508216 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.508579 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.508580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.508580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.508639 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.508816 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.508816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.508820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.508918 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.508969 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.509208 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.509270 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:06:42.509609 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.500910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46151 (local address 127.18.63.67:46743)
0808 07:06:42.501080 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.501116 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.501703 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.502000 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.502003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.502294 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:42.502825 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.502831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.503532 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.503533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.504366 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.504369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.504774 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.504775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.504898 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.505449 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:42.505458 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:42.506937 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:42.508290 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.508297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.508298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.508512 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.508680 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:42.508680 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:42.508680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:42.508764 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:42.508974 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:42.508979 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:42.509100 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.509181 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.509297 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260808 07:06:42.510083 24877 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46151
I20260808 07:06:42.510186 24877 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:42.510306 24877 heartbeater.cc:511] Master 127.18.63.126:46151 requested a full tablet report, sending...
I20260808 07:06:42.510685 24262 ts_manager.cc:194] Registered new tserver with Master: 3645c789851545038dfbc07be4414cc3 (127.18.63.67:43321)
I20260808 07:06:42.511068 24262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:46743
I20260808 07:06:42.517382 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:42.524477 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.518638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.18.63.126:46151)
0808 07:06:42.518705 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.518708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.518817 (+ 109us) server_negotiation.cc:407] Connection header received
0808 07:06:42.518903 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.518904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.518913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.518969 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.519528 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.520025 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.520744 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.520932 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.521002 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.522209 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.522211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.522211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:42.522219 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.523302 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.523647 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.523648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.523648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.523707 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:42.523905 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:42.523906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:42.523906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:42.524052 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.524154 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.524292 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.524376 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:06:42.525921 24262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:42.527426 24262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:42.533900 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.529739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43197 (local address 127.0.0.1:44190)
0808 07:06:42.530094 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.530150 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.530258 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.530581 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.530582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.530584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:42.530683 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.530685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.531599 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.531600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.532748 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.532750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.533599 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.533600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.533682 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.533696 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.533772 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.533823 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":32,"thread_start_us":95,"threads_started":1}
I20260808 07:06:42.533951 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.529964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.18.63.65:43197)
0808 07:06:42.530138 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.530144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.530223 (+ 79us) server_negotiation.cc:407] Connection header received
0808 07:06:42.530347 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.530350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.530370 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.530462 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:42.530775 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.531482 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.532830 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.533519 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.533595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.533654 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.533798 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.533844 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260808 07:06:42.534299 24891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.529954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40441 (local address 127.0.0.1:34578)
0808 07:06:42.530339 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.530348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.530393 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.530711 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.530712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.530714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:42.530750 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.530751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.531986 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.531987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.533000 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.533002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.534033 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.534035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.534101 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.534105 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.534193 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.534234 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":78,"threads_started":1}
I20260808 07:06:42.534341 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.530061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.18.63.66:40441)
0808 07:06:42.530237 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.530242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.530468 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:06:42.530518 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.530520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.530535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.530610 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:42.530816 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.531882 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.533073 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.533870 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.533939 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.533994 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.534214 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.534254 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260808 07:06:42.535020 24436 tablet_service.cc:1467] Processing CreateTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae (DEFAULT_TABLE table=test-table [id=a0e4d362fabf4b5ca244dda8b2a43220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:42.535018 24612 tablet_service.cc:1467] Processing CreateTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae (DEFAULT_TABLE table=test-table [id=a0e4d362fabf4b5ca244dda8b2a43220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:42.535324 24612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f9b7a4e64e4ab8bb332b6899094bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:42.535392 24436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f9b7a4e64e4ab8bb332b6899094bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:42.537926 24899 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Bootstrap starting.
I20260808 07:06:42.538172 24901 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: Bootstrap starting.
I20260808 07:06:42.538650 24899 tablet_bootstrap.cc:654] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:42.538803 24901 tablet_bootstrap.cc:654] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:42.538858 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.529830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.18.63.67:43321)
0808 07:06:42.530278 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.530282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.530295 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:42.530335 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.530337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.530352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.530448 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:42.530730 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.531874 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.534096 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.538428 (+ 4332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.538510 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.538566 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.538730 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.538776 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":116,"threads_started":1}
I20260808 07:06:42.539059 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.529739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:43321 (local address 127.0.0.1:45628)
0808 07:06:42.530158 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.530173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.530244 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.530563 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.530567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.530568 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:42.530664 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.530667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.533077 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.533077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.533924 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.533927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.538523 (+ 4596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.538524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.538621 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.538624 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.538766 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.538809 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":4,"thread_start_us":54,"threads_started":1}
I20260808 07:06:42.539343 24899 log.cc:824] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:42.539434 24901 log.cc:824] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:42.539781 24791 tablet_service.cc:1467] Processing CreateTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae (DEFAULT_TABLE table=test-table [id=a0e4d362fabf4b5ca244dda8b2a43220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:42.540122 24791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97f9b7a4e64e4ab8bb332b6899094bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:42.540244 24901 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: No bootstrap required, opened a new log
I20260808 07:06:42.540307 24901 ts_tablet_manager.cc:1397] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:42.540452 24899 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: No bootstrap required, opened a new log
I20260808 07:06:42.540504 24899 ts_tablet_manager.cc:1397] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:42.542897 24899 raft_consensus.cc:348] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.543078 24899 raft_consensus.cc:374] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:42.543120 24899 raft_consensus.cc:729] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd45936eefc4f40ab43e1c1b99e7abb, State: Initialized, Role: FOLLOWER
I20260808 07:06:42.543220 24905 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Bootstrap starting.
I20260808 07:06:42.543241 24899 consensus_queue.cc:260] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.543531 24899 ts_tablet_manager.cc:1428] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:42.543462 24901 raft_consensus.cc:348] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.543601 24700 heartbeater.cc:503] Master 127.18.63.126:46151 was elected leader, sending a full tablet report...
I20260808 07:06:42.543627 24901 raft_consensus.cc:374] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:42.543674 24901 raft_consensus.cc:729] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce22c2be60db41ae877f750a067d18df, State: Initialized, Role: FOLLOWER
I20260808 07:06:42.543876 24905 tablet_bootstrap.cc:654] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:42.543790 24901 consensus_queue.cc:260] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.544193 24901 ts_tablet_manager.cc:1428] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:06:42.544210 24518 heartbeater.cc:503] Master 127.18.63.126:46151 was elected leader, sending a full tablet report...
I20260808 07:06:42.544894 24905 log.cc:824] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:42.545943 24905 tablet_bootstrap.cc:492] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: No bootstrap required, opened a new log
I20260808 07:06:42.546028 24905 ts_tablet_manager.cc:1397] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:42.547372 24905 raft_consensus.cc:348] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.547515 24905 raft_consensus.cc:374] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:42.547564 24905 raft_consensus.cc:729] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3645c789851545038dfbc07be4414cc3, State: Initialized, Role: FOLLOWER
I20260808 07:06:42.547674 24905 consensus_queue.cc:260] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.547927 24905 ts_tablet_manager.cc:1428] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:06:42.547971 24877 heartbeater.cc:503] Master 127.18.63.126:46151 was elected leader, sending a full tablet report...
W20260808 07:06:42.586863 24701 tablet.cc:2369] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:42.603255 24911 raft_consensus.cc:482] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:42.603374 24911 raft_consensus.cc:504] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.603683 24911 leader_election.cc:288] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce22c2be60db41ae877f750a067d18df (127.18.63.65:43197), 7dd45936eefc4f40ab43e1c1b99e7abb (127.18.63.66:40441)
I20260808 07:06:42.606551 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.603876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43197 (local address 127.18.63.67:41177)
0808 07:06:42.603935 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.603948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.604006 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.604168 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.604170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.604172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:42.604256 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.604258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.604916 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.604917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.605753 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.605756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.606295 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.606296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.606375 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.606378 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.606442 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.606479 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:06:42.606551 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.603979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:41177 (local address 127.18.63.65:43197)
0808 07:06:42.604038 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.604040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.604049 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:42.604082 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.604083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.604093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.604160 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:42.604330 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.604833 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.605842 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.606214 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.606295 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.606304 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.606448 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.606483 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:06:42.606839 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.604018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40441 (local address 127.18.63.67:36071)
0808 07:06:42.604229 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.604238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.604308 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.604465 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.604465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.604467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:42.604555 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.604557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.605202 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.605203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.605965 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:42.605968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.606576 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.606578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:42.606665 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.606670 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.606733 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.606768 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":144,"threads_started":1}
I20260808 07:06:42.606848 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.604158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:36071 (local address 127.18.63.66:40441)
0808 07:06:42.604224 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.604226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.604303 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:06:42.604374 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.604375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.604399 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.604461 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:42.604645 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.605112 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.606070 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.606446 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.606555 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.606563 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.606743 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.606787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:06:42.606881 24465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f9b7a4e64e4ab8bb332b6899094bae" candidate_uuid: "3645c789851545038dfbc07be4414cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce22c2be60db41ae877f750a067d18df" is_pre_election: true
I20260808 07:06:42.607043 24465 raft_consensus.cc:2454] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3645c789851545038dfbc07be4414cc3 in term 0.
I20260808 07:06:42.607244 24644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f9b7a4e64e4ab8bb332b6899094bae" candidate_uuid: "3645c789851545038dfbc07be4414cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" is_pre_election: true
I20260808 07:06:42.607326 24729 leader_election.cc:302] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3645c789851545038dfbc07be4414cc3, ce22c2be60db41ae877f750a067d18df; no voters:
I20260808 07:06:42.607395 24644 raft_consensus.cc:2454] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3645c789851545038dfbc07be4414cc3 in term 0.
I20260808 07:06:42.607491 24911 raft_consensus.cc:2781] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:42.607574 24911 raft_consensus.cc:482] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:42.607610 24911 raft_consensus.cc:3037] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:42.608325 24911 raft_consensus.cc:504] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.608467 24911 leader_election.cc:288] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [CANDIDATE]: Term 1 election: Requested vote from peers ce22c2be60db41ae877f750a067d18df (127.18.63.65:43197), 7dd45936eefc4f40ab43e1c1b99e7abb (127.18.63.66:40441)
I20260808 07:06:42.608659 24465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f9b7a4e64e4ab8bb332b6899094bae" candidate_uuid: "3645c789851545038dfbc07be4414cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce22c2be60db41ae877f750a067d18df"
I20260808 07:06:42.608662 24644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97f9b7a4e64e4ab8bb332b6899094bae" candidate_uuid: "3645c789851545038dfbc07be4414cc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb"
I20260808 07:06:42.608741 24465 raft_consensus.cc:3037] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:42.608741 24644 raft_consensus.cc:3037] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:42.609694 24644 raft_consensus.cc:2454] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3645c789851545038dfbc07be4414cc3 in term 1.
I20260808 07:06:42.609694 24465 raft_consensus.cc:2454] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3645c789851545038dfbc07be4414cc3 in term 1.
I20260808 07:06:42.609872 24729 leader_election.cc:302] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3645c789851545038dfbc07be4414cc3, ce22c2be60db41ae877f750a067d18df; no voters:
I20260808 07:06:42.609984 24911 raft_consensus.cc:2781] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:42.610157 24911 raft_consensus.cc:686] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 1 LEADER]: Becoming Leader. State: Replica: 3645c789851545038dfbc07be4414cc3, State: Running, Role: LEADER
I20260808 07:06:42.610287 24911 consensus_queue.cc:237] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } }
I20260808 07:06:42.611192 24260 catalog_manager.cc:5495] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3645c789851545038dfbc07be4414cc3 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "3645c789851545038dfbc07be4414cc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3645c789851545038dfbc07be4414cc3" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:42.624607 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.622523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.18.63.67:43321)
0808 07:06:42.622577 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.622578 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.622586 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:42.622634 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.622635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.622650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.622708 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:42.622827 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.623261 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.623973 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:42.624146 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:42.624193 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:42.624330 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:42.624420 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:42.624478 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.624511 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.624526 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:42.626396 24644 raft_consensus.cc:1261] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 1 FOLLOWER]: Refusing update from remote peer 3645c789851545038dfbc07be4414cc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:42.626434 24465 raft_consensus.cc:1261] T 97f9b7a4e64e4ab8bb332b6899094bae P ce22c2be60db41ae877f750a067d18df [term 1 FOLLOWER]: Refusing update from remote peer 3645c789851545038dfbc07be4414cc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:42.626729 24911 consensus_queue.cc:1035] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd45936eefc4f40ab43e1c1b99e7abb" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:42.626874 24913 consensus_queue.cc:1035] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce22c2be60db41ae877f750a067d18df" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:42.629292 24921 mvcc.cc:205] Tried to move back new op lower bound from 7316163799555252224 to 7316163799492562944. Current Snapshot: MvccSnapshot[applied={T|T < 7316163799555252224}]
I20260808 07:06:42.629575 24922 mvcc.cc:205] Tried to move back new op lower bound from 7316163799555252224 to 7316163799492562944. Current Snapshot: MvccSnapshot[applied={T|T < 7316163799555252224}]
I20260808 07:06:42.631767 24260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:42.631868 24260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:42.632604 24260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 561b81fdb6de49cab91b7fbc41dcbea0: Sending DeleteTablet for 3 replicas of tablet 97f9b7a4e64e4ab8bb332b6899094bae
I20260808 07:06:42.632953 24436 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:42 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:44190
I20260808 07:06:42.633044 24791 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:42 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:45628
I20260808 07:06:42.633070 24612 tablet_service.cc:1514] Processing DeleteTablet for tablet 97f9b7a4e64e4ab8bb332b6899094bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:42 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:34578
I20260808 07:06:42.633455 24934 tablet_replica.cc:330] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: stopping tablet replica
I20260808 07:06:42.633526 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 24350
I20260808 07:06:42.633574 24934 raft_consensus.cc:2229] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:42.633701 24934 raft_consensus.cc:2258] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:42.638803 24936 tablet_replica.cc:330] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: stopping tablet replica
I20260808 07:06:42.638921 24936 raft_consensus.cc:2229] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:42.639020 24936 raft_consensus.cc:2258] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:42.639843 24934 ts_tablet_manager.cc:1905] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:42.642076 24934 ts_tablet_manager.cc:1918] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:42.642165 24934 log.cc:1196] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/97f9b7a4e64e4ab8bb332b6899094bae
I20260808 07:06:42.642458 24934 ts_tablet_manager.cc:1939] T 97f9b7a4e64e4ab8bb332b6899094bae P 3645c789851545038dfbc07be4414cc3: Deleting consensus metadata
I20260808 07:06:42.642957 24241 catalog_manager.cc:4841] TS 3645c789851545038dfbc07be4414cc3 (127.18.63.67:43321): tablet 97f9b7a4e64e4ab8bb332b6899094bae (table test-table [id=a0e4d362fabf4b5ca244dda8b2a43220]) successfully deleted
I20260808 07:06:42.643198 24936 ts_tablet_manager.cc:1905] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:42.645169 24936 ts_tablet_manager.cc:1918] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:42.645254 24936 log.cc:1196] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/97f9b7a4e64e4ab8bb332b6899094bae
I20260808 07:06:42.645597 24936 ts_tablet_manager.cc:1939] T 97f9b7a4e64e4ab8bb332b6899094bae P 7dd45936eefc4f40ab43e1c1b99e7abb: Deleting consensus metadata
I20260808 07:06:42.646070 24240 catalog_manager.cc:4841] TS 7dd45936eefc4f40ab43e1c1b99e7abb (127.18.63.66:40441): tablet 97f9b7a4e64e4ab8bb332b6899094bae (table test-table [id=a0e4d362fabf4b5ca244dda8b2a43220]) successfully deleted
W20260808 07:06:42.646479 24240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:43197 (error 108)
I20260808 07:06:42.647053 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.646906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43197 (local address 127.0.0.1:44192)
0808 07:06:42.646970 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.647011 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:06:42.647152 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 24525
W20260808 07:06:42.652510 24240 catalog_manager.cc:4584] TS ce22c2be60db41ae877f750a067d18df (127.18.63.65:43197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97f9b7a4e64e4ab8bb332b6899094bae: Network error: Client connection negotiation failed: client connection to 127.18.63.65:43197: connect: Connection refused (error 111)
I20260808 07:06:42.658633 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 24706
I20260808 07:06:42.666343 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 24215
2026-08-08T07:06:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2137 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T07:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:42Z Disabled control of system clock
I20260808 07:06:42.698256 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42593
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:06:42.781289 24945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:42.781507 24945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:42.781527 24945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:42.782680 24945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:42.782720 24945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:42.782733 24945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:42.782747 24945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:42.782759 24945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:42.784040 24945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:42.784236 24945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:42.784484 24945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:42.789322 24956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.789381 24954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.789423 24945 server_base.cc:1034] running on GCE node
W20260808 07:06:42.789391 24953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.789830 24945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:42.790127 24945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:42.791297 24945 hybrid_clock.cc:648] HybridClock initialized: now 1786172802791285 us; error 51 us; skew 500 ppm
I20260808 07:06:42.792609 24945 webserver.cc:460] Webserver started at http://127.18.63.126:33173/ using document root <none> and password file <none>
I20260808 07:06:42.792883 24945 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:42.792953 24945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:42.793102 24945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:42.794224 24945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "987bdfd74af94dc0aae43901d7548ed7"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "f9c68aa4bcbbd72cb2dcb6951f7d27c2"
server_key_iv: "32783e51732ddabce87259d4ed125702"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.794587 24945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "987bdfd74af94dc0aae43901d7548ed7"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "f9c68aa4bcbbd72cb2dcb6951f7d27c2"
server_key_iv: "32783e51732ddabce87259d4ed125702"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.795879 24945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:42.796578 24962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.796751 24945 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:42.796852 24945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "987bdfd74af94dc0aae43901d7548ed7"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "f9c68aa4bcbbd72cb2dcb6951f7d27c2"
server_key_iv: "32783e51732ddabce87259d4ed125702"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.796929 24945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:42.800295 24945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:42.800556 24945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:42.800693 24945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:42.804721 24945 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:42593
I20260808 07:06:42.804752 25036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:42593 every 8 connection(s)
I20260808 07:06:42.805070 24945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:42.805827 25037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:42.808502 25037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: Bootstrap starting.
I20260808 07:06:42.809041 25037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:42.809295 25037 log.cc:824] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:42.810096 25037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: No bootstrap required, opened a new log
I20260808 07:06:42.811645 25037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } }
I20260808 07:06:42.811776 25037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:42.811798 25037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 987bdfd74af94dc0aae43901d7548ed7, State: Initialized, Role: FOLLOWER
I20260808 07:06:42.811929 25037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } }
I20260808 07:06:42.812006 25037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:42.812027 25037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:42.812055 25037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:42.812572 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 24945
I20260808 07:06:42.812680 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:42.812665 25037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } }
I20260808 07:06:42.812765 25037 leader_election.cc:302] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 987bdfd74af94dc0aae43901d7548ed7; no voters:
I20260808 07:06:42.812883 18685 external_mini_cluster.cc:1428] Setting key d3eca08e9691fd0698f69cbf35570de8
I20260808 07:06:42.812932 25037 leader_election.cc:288] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:42.812997 25044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:42.813194 25037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:42.813201 25044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [term 1 LEADER]: Becoming Leader. State: Replica: 987bdfd74af94dc0aae43901d7548ed7, State: Running, Role: LEADER
I20260808 07:06:42.813323 25044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } }
I20260808 07:06:42.813663 25044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 987bdfd74af94dc0aae43901d7548ed7. Latest consensus state: current_term: 1 leader_uuid: "987bdfd74af94dc0aae43901d7548ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } } }
I20260808 07:06:42.813760 25044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:42.813673 25046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "987bdfd74af94dc0aae43901d7548ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "987bdfd74af94dc0aae43901d7548ed7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42593 } } }
I20260808 07:06:42.813899 25046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:42.813961 25051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:42.814502 25051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:42.815735 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.813503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35344 (local address 127.18.63.126:42593)
0808 07:06:42.813866 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.813877 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.813917 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:06:42.814044 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.814050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.814094 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.814262 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.815034 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.815038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.815039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:42.815043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.815088 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:42.815103 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.815162 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.815350 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.815409 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":74,"threads_started":1}
I20260808 07:06:42.816888 25051 catalog_manager.cc:1347] Generated new cluster ID: 7c2d1196c23d4b89ac58d119a465b00a
I20260808 07:06:42.816946 25051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:42.827865 25051 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:42.828042 25051 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:42.834627 25051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: Generated new TSK 0
I20260808 07:06:42.834820 25051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:42.841497 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--builtin_ntp_servers=127.18.63.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:06:42.927470 25071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:42.927698 25071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:42.927735 25071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:42.928903 25071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:42.928966 25071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:42.929006 25071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:42.930431 25071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:42.930709 25071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:42.930997 25071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:42.937047 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.937039 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:42.937355 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:42.937317 25071 server_base.cc:1034] running on GCE node
I20260808 07:06:42.937583 25071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:42.937858 25071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:42.939031 25071 hybrid_clock.cc:648] HybridClock initialized: now 1786172802939009 us; error 41 us; skew 500 ppm
I20260808 07:06:42.940222 25071 webserver.cc:460] Webserver started at http://127.18.63.65:35579/ using document root <none> and password file <none>
I20260808 07:06:42.940459 25071 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:42.940573 25071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:42.940719 25071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:42.941880 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "949b906719ec4dc7985bb9cf7bd1e4bc"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "2f7e824dcf361e31a3bd089896d8a2b6"
server_key_iv: "9262c77bcb2dc99b20329f49eb57355a"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.942250 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "949b906719ec4dc7985bb9cf7bd1e4bc"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "2f7e824dcf361e31a3bd089896d8a2b6"
server_key_iv: "9262c77bcb2dc99b20329f49eb57355a"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.943635 25071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:42.944376 25089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.944574 25071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:42.944690 25071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "949b906719ec4dc7985bb9cf7bd1e4bc"
format_stamp: "Formatted at 2026-08-08 07:06:42 on dist-test-slave-bw88"
server_key: "2f7e824dcf361e31a3bd089896d8a2b6"
server_key_iv: "9262c77bcb2dc99b20329f49eb57355a"
server_key_version: "encryptionkey@0"
I20260808 07:06:42.944764 25071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:42.948122 25071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:42.948383 25071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:42.948523 25071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:42.948747 25071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:42.949075 25071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:42.949129 25071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.949163 25071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:42.949193 25071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:42.955380 25071 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:40177
I20260808 07:06:42.955430 25245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:40177 every 8 connection(s)
I20260808 07:06:42.955801 25071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:42.957261 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25071
I20260808 07:06:42.957408 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:42.957602 18685 external_mini_cluster.cc:1428] Setting key 0554a867e51c341b899722b2bcf2889c
I20260808 07:06:42.958570 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--builtin_ntp_servers=127.18.63.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 07:06:42.959034 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.956509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:51187 (local address 127.18.63.126:42593)
0808 07:06:42.956572 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:42.956574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:42.956827 (+ 253us) server_negotiation.cc:407] Connection header received
0808 07:06:42.957321 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:42.957323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:42.957332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:42.957428 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:42.958734 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:42.958736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:42.958736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:42.958741 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:42.958760 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:42.958766 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:42.958826 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:42.958920 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:06:42.958937 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:42.959144 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:42.956506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42593 (local address 127.18.63.65:51187)
0808 07:06:42.956709 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:42.956742 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:42.957216 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:42.957506 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:42.957510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:42.957521 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:42.958558 (+ 1037us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:42.958559 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:42.958586 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:42.958633 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:42.958836 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:42.958842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:42.958844 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:42.958914 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:06:42.958963 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260808 07:06:42.959679 25246 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42593
I20260808 07:06:42.959781 25246 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:42.959933 25246 heartbeater.cc:511] Master 127.18.63.126:42593 requested a full tablet report, sending...
I20260808 07:06:42.960527 24987 ts_manager.cc:194] Registered new tserver with Master: 949b906719ec4dc7985bb9cf7bd1e4bc (127.18.63.65:40177)
W20260808 07:06:43.042889 25249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:43.043120 25249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:43.043164 25249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:43.044284 25249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:43.044318 25249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:43.044343 25249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:43.045713 25249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:43.045914 25249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:43.046166 25249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:43.051415 25257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.051420 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.051415 25258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.051764 25249 server_base.cc:1034] running on GCE node
I20260808 07:06:43.051968 25249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:43.052213 25249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:43.053401 25249 hybrid_clock.cc:648] HybridClock initialized: now 1786172803053384 us; error 27 us; skew 500 ppm
I20260808 07:06:43.054669 25249 webserver.cc:460] Webserver started at http://127.18.63.66:35207/ using document root <none> and password file <none>
I20260808 07:06:43.054924 25249 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:43.054994 25249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:43.055151 25249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:43.056206 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "107f191d87c346afb6158b6601b07c89"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "e5f529f89f6806d3555ed1f9d6d14603"
server_key_iv: "71c26e1937ddfe7946c2c8b819b4e3dc"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.056577 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "107f191d87c346afb6158b6601b07c89"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "e5f529f89f6806d3555ed1f9d6d14603"
server_key_iv: "71c26e1937ddfe7946c2c8b819b4e3dc"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.058017 25249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:43.058769 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.058969 25249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:43.059072 25249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "107f191d87c346afb6158b6601b07c89"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "e5f529f89f6806d3555ed1f9d6d14603"
server_key_iv: "71c26e1937ddfe7946c2c8b819b4e3dc"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.059145 25249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:43.062569 25249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:43.062834 25249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:43.062969 25249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:43.063177 25249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:43.063481 25249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:43.063542 25249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.063582 25249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:43.063611 25249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.069602 25249 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:42237
I20260808 07:06:43.069638 25417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:42237 every 8 connection(s)
I20260808 07:06:43.069959 25249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:43.072799 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.070696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:37499 (local address 127.18.63.126:42593)
0808 07:06:43.070760 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.070763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.071015 (+ 252us) server_negotiation.cc:407] Connection header received
0808 07:06:43.071466 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.071466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.071471 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.071531 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.072595 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.072597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.072597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.072601 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.072617 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.072621 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.072676 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.072731 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.072744 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:43.072980 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.070699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42593 (local address 127.18.63.66:37499)
0808 07:06:43.070879 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.070919 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.071383 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.071581 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.071584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.071592 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.072445 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.072446 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.072474 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.072511 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.072672 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.072674 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.072676 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.072726 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.072772 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260808 07:06:43.073591 25418 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42593
I20260808 07:06:43.073679 25418 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:43.073793 25418 heartbeater.cc:511] Master 127.18.63.126:42593 requested a full tablet report, sending...
I20260808 07:06:43.074115 24987 ts_manager.cc:194] Registered new tserver with Master: 107f191d87c346afb6158b6601b07c89 (127.18.63.66:42237)
I20260808 07:06:43.074406 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25249
I20260808 07:06:43.074512 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:43.074708 18685 external_mini_cluster.cc:1428] Setting key cfdf03d2b5422cf97f74fbd3fcfb6c29
I20260808 07:06:43.075551 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--builtin_ntp_servers=127.18.63.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 07:06:43.163666 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:43.163889 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:43.163920 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:43.165148 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:43.165194 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:43.165223 25421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:43.166744 25421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:43.166970 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:43.167264 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:43.172839 25429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.172832 25433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.172863 25428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.173058 25421 server_base.cc:1034] running on GCE node
I20260808 07:06:43.173265 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:43.173560 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:43.174711 25421 hybrid_clock.cc:648] HybridClock initialized: now 1786172803174695 us; error 33 us; skew 500 ppm
I20260808 07:06:43.175887 25421 webserver.cc:460] Webserver started at http://127.18.63.67:39023/ using document root <none> and password file <none>
I20260808 07:06:43.176115 25421 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:43.176157 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:43.176252 25421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:43.177310 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "3ac41b55916a40e68e572b4144427bef"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "5a0f464e9d1882aa7645095cc1210a28"
server_key_iv: "c85512c1e9df44a7af234d330a833a6e"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.177697 25421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "3ac41b55916a40e68e572b4144427bef"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "5a0f464e9d1882aa7645095cc1210a28"
server_key_iv: "c85512c1e9df44a7af234d330a833a6e"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.179118 25421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:43.179838 25439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.179999 25421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:43.180104 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "3ac41b55916a40e68e572b4144427bef"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "5a0f464e9d1882aa7645095cc1210a28"
server_key_iv: "c85512c1e9df44a7af234d330a833a6e"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.180176 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:43.183660 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:43.183925 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:43.184068 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:43.184288 25421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:43.184604 25421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:43.184657 25421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.184697 25421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:43.184725 25421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.191004 25421 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:39699
I20260808 07:06:43.191059 25581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:39699 every 8 connection(s)
I20260808 07:06:43.191377 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:43.194252 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.192062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:34177 (local address 127.18.63.126:42593)
0808 07:06:43.192129 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.192131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.192364 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:06:43.192837 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.192838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.192844 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.192898 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.194054 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.194055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.194056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.194059 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.194071 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.194075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.194121 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.194183 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.194197 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:06:43.194450 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.192047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42593 (local address 127.18.63.67:34177)
0808 07:06:43.192232 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.192281 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.192749 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.192949 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.192952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.192959 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.193921 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.193922 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.193950 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.193977 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.194120 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.194129 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.194131 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.194182 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.194228 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260808 07:06:43.194983 25583 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42593
I20260808 07:06:43.195080 25583 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:43.195232 25583 heartbeater.cc:511] Master 127.18.63.126:42593 requested a full tablet report, sending...
I20260808 07:06:43.195623 24987 ts_manager.cc:194] Registered new tserver with Master: 3ac41b55916a40e68e572b4144427bef (127.18.63.67:39699)
I20260808 07:06:43.201390 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25421
I20260808 07:06:43.201496 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:43.201687 18685 external_mini_cluster.cc:1428] Setting key 70256c64b732a8805c6f2376eb0b2002
I20260808 07:06:43.202526 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:06:43.204972 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.203860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.18.63.126:42593)
0808 07:06:43.203915 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.203916 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.204053 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:06:43.204138 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.204139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.204144 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.204209 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.204762 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.204763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.204763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.204767 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.204781 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.204785 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.204831 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.204885 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.204901 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:43.206482 24987 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:43.208014 24987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:43.211542 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40177 (local address 127.0.0.1:45924)
0808 07:06:43.210155 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.210194 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.210261 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.210614 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.210621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.210627 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.211098 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.211098 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.211117 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.211140 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.211403 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.211411 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.211414 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.211460 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.211474 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260808 07:06:43.211905 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.18.63.65:40177)
0808 07:06:43.210455 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.210461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.210473 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:43.210503 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.210506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.210516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.210603 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.211237 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.211246 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.211246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.211251 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.211289 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.211320 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.211751 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.211784 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.211801 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":120,"threads_started":1}
I20260808 07:06:43.212046 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:39699 (local address 127.0.0.1:44604)
0808 07:06:43.210191 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.210202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.210260 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.211050 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.211051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.211054 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.211486 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.211487 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.211495 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.211508 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.211914 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.211915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.211915 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.211959 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.211972 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":112,"threads_started":1}
I20260808 07:06:43.212169 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42237 (local address 127.0.0.1:35498)
0808 07:06:43.210410 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.210420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.210452 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.211324 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.211324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.211327 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.211775 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.211776 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.211781 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.211795 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.212029 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.212030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.212031 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.212078 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.212092 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260808 07:06:43.212368 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35498 (local address 127.18.63.66:42237)
0808 07:06:43.210903 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.210907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.210917 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:43.210944 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.210947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.210956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.211026 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.211864 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.211874 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.211874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.211879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.211911 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.211940 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.212000 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.212269 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.212288 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260808 07:06:43.212666 25154 tablet_service.cc:1467] Processing CreateTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 (DEFAULT_TABLE table=test-table [id=061b8a3779014fc8a32f9dcfec3085da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:43.213032 25154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3453f370afc34e6a916a2dcc471d5ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:43.213013 25331 tablet_service.cc:1467] Processing CreateTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 (DEFAULT_TABLE table=test-table [id=061b8a3779014fc8a32f9dcfec3085da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:43.213413 25331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3453f370afc34e6a916a2dcc471d5ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:43.213459 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.210441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.18.63.67:39699)
0808 07:06:43.210632 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.210637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.210647 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:43.210669 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.210671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.210679 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.211392 (+ 713us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.211804 (+ 412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.211808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.211808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.211811 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.211841 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.211857 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.211921 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.211978 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.211994 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260808 07:06:43.214072 25494 tablet_service.cc:1467] Processing CreateTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 (DEFAULT_TABLE table=test-table [id=061b8a3779014fc8a32f9dcfec3085da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:43.214455 25494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3453f370afc34e6a916a2dcc471d5ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:43.216420 25599 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Bootstrap starting.
I20260808 07:06:43.216770 25600 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: Bootstrap starting.
I20260808 07:06:43.217127 25599 tablet_bootstrap.cc:654] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:43.217478 25600 tablet_bootstrap.cc:654] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:43.217504 25599 log.cc:824] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:43.217796 25601 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Bootstrap starting.
I20260808 07:06:43.217823 25600 log.cc:824] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:43.218510 25601 tablet_bootstrap.cc:654] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:43.218521 25599 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: No bootstrap required, opened a new log
I20260808 07:06:43.218595 25599 ts_tablet_manager.cc:1397] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:43.218807 25600 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: No bootstrap required, opened a new log
I20260808 07:06:43.218894 25600 ts_tablet_manager.cc:1397] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:43.218883 25601 log.cc:824] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:43.219842 25601 tablet_bootstrap.cc:492] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: No bootstrap required, opened a new log
I20260808 07:06:43.219919 25601 ts_tablet_manager.cc:1397] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:43.221055 25600 raft_consensus.cc:348] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.221221 25600 raft_consensus.cc:374] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:43.221271 25600 raft_consensus.cc:729] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949b906719ec4dc7985bb9cf7bd1e4bc, State: Initialized, Role: FOLLOWER
I20260808 07:06:43.221254 25599 raft_consensus.cc:348] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.221448 25599 raft_consensus.cc:374] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:43.221489 25599 raft_consensus.cc:729] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 107f191d87c346afb6158b6601b07c89, State: Initialized, Role: FOLLOWER
I20260808 07:06:43.221444 25600 consensus_queue.cc:260] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.221627 25599 consensus_queue.cc:260] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.221908 25600 ts_tablet_manager.cc:1428] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:06:43.221920 25418 heartbeater.cc:503] Master 127.18.63.126:42593 was elected leader, sending a full tablet report...
I20260808 07:06:43.222234 25246 heartbeater.cc:503] Master 127.18.63.126:42593 was elected leader, sending a full tablet report...
I20260808 07:06:43.222491 25599 ts_tablet_manager.cc:1428] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260808 07:06:43.222694 25601 raft_consensus.cc:348] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.222855 25601 raft_consensus.cc:374] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:43.222909 25601 raft_consensus.cc:729] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac41b55916a40e68e572b4144427bef, State: Initialized, Role: FOLLOWER
I20260808 07:06:43.223022 25601 consensus_queue.cc:260] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.223310 25601 ts_tablet_manager.cc:1428] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:06:43.223376 25583 heartbeater.cc:503] Master 127.18.63.126:42593 was elected leader, sending a full tablet report...
I20260808 07:06:43.260792 25607 raft_consensus.cc:482] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:43.260927 25607 raft_consensus.cc:504] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.261390 25607 leader_election.cc:288] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 949b906719ec4dc7985bb9cf7bd1e4bc (127.18.63.65:40177), 107f191d87c346afb6158b6601b07c89 (127.18.63.66:42237)
I20260808 07:06:43.262832 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.261572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40177 (local address 127.18.63.67:53403)
0808 07:06:43.261638 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.261653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.261708 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.261894 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.261895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.261898 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.262429 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.262429 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.262439 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.262454 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.262687 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.262688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.262689 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.262762 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.262776 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:06:43.262984 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.261671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:39579 (local address 127.18.63.66:42237)
0808 07:06:43.261720 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.261722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.262086 (+ 364us) server_negotiation.cc:407] Connection header received
0808 07:06:43.262123 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.262124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.262132 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.262185 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.262743 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.262744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.262744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.262749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.262769 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.262774 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.262825 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.262875 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.262891 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:43.263026 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.261718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:53403 (local address 127.18.63.65:40177)
0808 07:06:43.261776 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.261777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.261786 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:43.261810 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.261811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.261816 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.261868 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.262575 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.262577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.262578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.262583 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.262604 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.262609 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.262677 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.262948 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.262965 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:43.263224 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.261572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42237 (local address 127.18.63.67:39579)
0808 07:06:43.261816 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.261824 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.261878 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.262210 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.262211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.262214 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:43.262616 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:06:43.262617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:43.262624 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:06:43.262638 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:43.262824 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:43.262825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:43.262825 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 07:06:43.262877 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:06:43.262890 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":13,"thread_start_us":120,"threads_started":1}
I20260808 07:06:43.263345 25182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3453f370afc34e6a916a2dcc471d5ae4" candidate_uuid: "3ac41b55916a40e68e572b4144427bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" is_pre_election: true
I20260808 07:06:43.263511 25182 raft_consensus.cc:2454] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ac41b55916a40e68e572b4144427bef in term 0.
I20260808 07:06:43.263623 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3453f370afc34e6a916a2dcc471d5ae4" candidate_uuid: "3ac41b55916a40e68e572b4144427bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107f191d87c346afb6158b6601b07c89" is_pre_election: true
I20260808 07:06:43.263784 25356 raft_consensus.cc:2454] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ac41b55916a40e68e572b4144427bef in term 0.
I20260808 07:06:43.263792 25441 leader_election.cc:302] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac41b55916a40e68e572b4144427bef, 949b906719ec4dc7985bb9cf7bd1e4bc; no voters:
I20260808 07:06:43.263907 25607 raft_consensus.cc:2781] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:43.263991 25607 raft_consensus.cc:482] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:43.264019 25607 raft_consensus.cc:3037] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:43.264773 25607 raft_consensus.cc:504] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.264912 25607 leader_election.cc:288] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [CANDIDATE]: Term 1 election: Requested vote from peers 949b906719ec4dc7985bb9cf7bd1e4bc (127.18.63.65:40177), 107f191d87c346afb6158b6601b07c89 (127.18.63.66:42237)
I20260808 07:06:43.265062 25182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3453f370afc34e6a916a2dcc471d5ae4" candidate_uuid: "3ac41b55916a40e68e572b4144427bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc"
I20260808 07:06:43.265075 25356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3453f370afc34e6a916a2dcc471d5ae4" candidate_uuid: "3ac41b55916a40e68e572b4144427bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "107f191d87c346afb6158b6601b07c89"
I20260808 07:06:43.265166 25182 raft_consensus.cc:3037] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:43.265161 25356 raft_consensus.cc:3037] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:43.265956 25182 raft_consensus.cc:2454] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ac41b55916a40e68e572b4144427bef in term 1.
I20260808 07:06:43.266003 25356 raft_consensus.cc:2454] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ac41b55916a40e68e572b4144427bef in term 1.
I20260808 07:06:43.266177 25440 leader_election.cc:302] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 107f191d87c346afb6158b6601b07c89, 3ac41b55916a40e68e572b4144427bef; no voters:
I20260808 07:06:43.266274 25607 raft_consensus.cc:2781] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:43.266474 25607 raft_consensus.cc:686] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 1 LEADER]: Becoming Leader. State: Replica: 3ac41b55916a40e68e572b4144427bef, State: Running, Role: LEADER
I20260808 07:06:43.266579 25607 consensus_queue.cc:237] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } }
I20260808 07:06:43.267421 24984 catalog_manager.cc:5495] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ac41b55916a40e68e572b4144427bef (127.18.63.67). New cstate: current_term: 1 leader_uuid: "3ac41b55916a40e68e572b4144427bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ac41b55916a40e68e572b4144427bef" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:43.285804 25597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.284804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44618 (local address 127.18.63.67:39699)
0808 07:06:43.284861 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.284863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.284873 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:43.284938 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.284939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.284950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.285003 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.285627 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.285629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.285629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.285633 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.285647 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.285652 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.285699 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.285743 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.285754 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:06:43.287534 25182 raft_consensus.cc:1261] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc [term 1 FOLLOWER]: Refusing update from remote peer 3ac41b55916a40e68e572b4144427bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:43.287536 25356 raft_consensus.cc:1261] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 1 FOLLOWER]: Refusing update from remote peer 3ac41b55916a40e68e572b4144427bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:43.287901 25609 consensus_queue.cc:1035] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "107f191d87c346afb6158b6601b07c89" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:43.288017 25607 consensus_queue.cc:1035] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "949b906719ec4dc7985bb9cf7bd1e4bc" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:43.290778 25617 mvcc.cc:205] Tried to move back new op lower bound from 7316163802263285760 to 7316163802180587520. Current Snapshot: MvccSnapshot[applied={T|T < 7316163802263285760}]
I20260808 07:06:43.292919 24984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:43.293005 24984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:43.293794 24984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 987bdfd74af94dc0aae43901d7548ed7: Sending DeleteTablet for 3 replicas of tablet 3453f370afc34e6a916a2dcc471d5ae4
I20260808 07:06:43.294113 25154 tablet_service.cc:1514] Processing DeleteTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:43 UTC) from {username='slave'} at 127.0.0.1:45924
I20260808 07:06:43.294162 25331 tablet_service.cc:1514] Processing DeleteTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:43 UTC) from {username='slave'} at 127.0.0.1:35498
I20260808 07:06:43.294188 25494 tablet_service.cc:1514] Processing DeleteTablet for tablet 3453f370afc34e6a916a2dcc471d5ae4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:43 UTC) from {username='slave'} at 127.0.0.1:44604
I20260808 07:06:43.294394 25634 tablet_replica.cc:330] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: stopping tablet replica
I20260808 07:06:43.294430 25630 tablet_replica.cc:330] T 3453f370afc34e6a916a2dcc471d5ae4 P 949b906719ec4dc7985bb9cf7bd1e4bc: stopping tablet replica
I20260808 07:06:43.294505 25634 raft_consensus.cc:2229] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:43.294638 25631 tablet_replica.cc:330] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: stopping tablet replica
I20260808 07:06:43.294690 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25071
I20260808 07:06:43.294739 25631 raft_consensus.cc:2229] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:43.294869 25631 raft_consensus.cc:2258] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:43.297307 25631 ts_tablet_manager.cc:1905] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:43.299820 25631 ts_tablet_manager.cc:1918] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:43.299916 25631 log.cc:1196] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/3453f370afc34e6a916a2dcc471d5ae4
I20260808 07:06:43.300377 25631 ts_tablet_manager.cc:1939] T 3453f370afc34e6a916a2dcc471d5ae4 P 3ac41b55916a40e68e572b4144427bef: Deleting consensus metadata
I20260808 07:06:43.300972 24964 catalog_manager.cc:4841] TS 3ac41b55916a40e68e572b4144427bef (127.18.63.67:39699): tablet 3453f370afc34e6a916a2dcc471d5ae4 (table test-table [id=061b8a3779014fc8a32f9dcfec3085da]) successfully deleted
I20260808 07:06:43.308223 25634 raft_consensus.cc:2258] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:43.308918 25634 ts_tablet_manager.cc:1905] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:43.311157 25634 ts_tablet_manager.cc:1918] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:43.311251 25634 log.cc:1196] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/3453f370afc34e6a916a2dcc471d5ae4
I20260808 07:06:43.311525 25634 ts_tablet_manager.cc:1939] T 3453f370afc34e6a916a2dcc471d5ae4 P 107f191d87c346afb6158b6601b07c89: Deleting consensus metadata
I20260808 07:06:43.312041 24964 catalog_manager.cc:4841] TS 107f191d87c346afb6158b6601b07c89 (127.18.63.66:42237): tablet 3453f370afc34e6a916a2dcc471d5ae4 (table test-table [id=061b8a3779014fc8a32f9dcfec3085da]) successfully deleted
W20260808 07:06:43.315279 24965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:40177 (error 108)
I20260808 07:06:43.315546 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25249
I20260808 07:06:43.328056 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.327836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40177 (local address 127.0.0.1:45938)
0808 07:06:43.327921 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.328005 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:40177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 07:06:43.328403 24965 catalog_manager.cc:4584] TS 949b906719ec4dc7985bb9cf7bd1e4bc (127.18.63.65:40177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3453f370afc34e6a916a2dcc471d5ae4: Network error: Client connection negotiation failed: client connection to 127.18.63.65:40177: connect: Connection refused (error 111)
I20260808 07:06:43.336072 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25421
I20260808 07:06:43.354449 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 24945
2026-08-08T07:06:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (690 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 07:06:43.380919 18685 mini_oidc.cc:152] Starting JWKS server
I20260808 07:06:43.382215 18685 webserver.cc:460] Webserver started at https://127.0.0.1:33667/ using document root <none> and password file <none>
I20260808 07:06:43.382325 18685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:06:43.382498 18685 webserver.cc:460] Webserver started at http://127.0.0.1:35463/ using document root <none> and password file <none>
2026-08-08T07:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:43Z Disabled control of system clock
I20260808 07:06:43.391517 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33573
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33573
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33667/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/testchainca.cert with env {}
W20260808 07:06:43.473877 25641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:43.474081 25641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:43.474102 25641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:43.475183 25641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:43.475215 25641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:43.475229 25641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:43.475241 25641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:43.475253 25641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:06:43.475265 25641 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:06:43.475277 25641 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33667/jwks.json
I20260808 07:06:43.476818 25641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33667/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:43.477012 25641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:43.477182 25641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:43.482213 25647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.482236 25653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.482440 25641 server_base.cc:1034] running on GCE node
W20260808 07:06:43.482225 25649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.482785 25641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:43.483072 25641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:43.484251 25641 hybrid_clock.cc:648] HybridClock initialized: now 1786172803484241 us; error 32 us; skew 500 ppm
I20260808 07:06:43.485548 25641 webserver.cc:460] Webserver started at http://127.18.63.126:43431/ using document root <none> and password file <none>
I20260808 07:06:43.485780 25641 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:43.485845 25641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:43.485981 25641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:43.487082 25641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "5308c09b37e0466f95a8e25c46cc6862"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "121591aba4dc680417bdaf8530358594"
server_key_iv: "a585b241831f9bdbfd8dcc2711d960f8"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.487454 25641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "5308c09b37e0466f95a8e25c46cc6862"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "121591aba4dc680417bdaf8530358594"
server_key_iv: "a585b241831f9bdbfd8dcc2711d960f8"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.488816 25641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:43.489593 25663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.489768 25641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:43.489867 25641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "5308c09b37e0466f95a8e25c46cc6862"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "121591aba4dc680417bdaf8530358594"
server_key_iv: "a585b241831f9bdbfd8dcc2711d960f8"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.489943 25641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:43.510676 25641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:43.511022 25641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:43.511202 25641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:43.515358 25641 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33573
I20260808 07:06:43.515399 25735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33573 every 8 connection(s)
I20260808 07:06:43.515776 25641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:43.516101 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25641
I20260808 07:06:43.516206 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:43.516366 18685 external_mini_cluster.cc:1428] Setting key 383fbb818ef6422e3d9785af1a1fafbe
I20260808 07:06:43.517024 25736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:43.520299 25736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862: Bootstrap starting.
I20260808 07:06:43.521003 25736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:43.521425 25736 log.cc:824] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:43.522495 25736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862: No bootstrap required, opened a new log
I20260808 07:06:43.522701 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.516875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.18.63.126:33573)
0808 07:06:43.517129 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.517139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.517171 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:06:43.517823 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.517832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.517915 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.518065 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.518280 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.519655 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.520445 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.521044 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.521190 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.521959 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.521963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.521963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.522174 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.522223 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.522239 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.522310 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.522380 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.522460 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
I20260808 07:06:43.523649 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:43.524847 25736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } }
I20260808 07:06:43.525012 25736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:43.525061 25736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5308c09b37e0466f95a8e25c46cc6862, State: Initialized, Role: FOLLOWER
I20260808 07:06:43.525182 25736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } }
I20260808 07:06:43.525270 25736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:43.525312 25736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:43.525396 25736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:43.525926 25736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } }
I20260808 07:06:43.526048 25736 leader_election.cc:302] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5308c09b37e0466f95a8e25c46cc6862; no voters:
I20260808 07:06:43.526206 25736 leader_election.cc:288] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:43.526333 25742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:43.526473 25736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:43.526505 25742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [term 1 LEADER]: Becoming Leader. State: Replica: 5308c09b37e0466f95a8e25c46cc6862, State: Running, Role: LEADER
I20260808 07:06:43.526583 25742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } }
I20260808 07:06:43.526994 25744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5308c09b37e0466f95a8e25c46cc6862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } } }
I20260808 07:06:43.527021 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5308c09b37e0466f95a8e25c46cc6862. Latest consensus state: current_term: 1 leader_uuid: "5308c09b37e0466f95a8e25c46cc6862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308c09b37e0466f95a8e25c46cc6862" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33573 } } }
I20260808 07:06:43.527128 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:43.527289 25744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:43.527361 25760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:43.527868 25760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:43.529392 25760 catalog_manager.cc:1347] Generated new cluster ID: 51ddb7e53c8c45ff9776682a5778af37
I20260808 07:06:43.529444 25760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:43.551007 25760 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:43.551674 25760 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:43.553992 25760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5308c09b37e0466f95a8e25c46cc6862: Generated new TSK 0
I20260808 07:06:43.554183 25760 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:06:43.586755 25780 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 07:06:43.586920 25780 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:06:43.587113 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.571742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.18.63.126:33573)
0808 07:06:43.571807 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.571810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.571958 (+ 148us) server_negotiation.cc:407] Connection header received
0808 07:06:43.572084 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.572085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.572096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.572159 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:06:43.572320 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.572821 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.573953 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.574141 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.574189 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.574424 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:06:43.586829 (+ 12405us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:06:43.587032 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:06:43.587181 25740 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41708: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 07:06:43.587729 25779 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:43.571683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33573 (local address 127.0.0.1:41708)
0808 07:06:43.571882 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.571895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.571975 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.572185 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.572186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.572188 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:06:43.572259 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:43.572261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.572913 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.572914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:43.573865 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:43.573869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.574221 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.574222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:43.574288 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.574311 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:06:43.587133 (+ 12822us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 07:06:43.587292 (+ 159us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.63.126:33573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.63.126:33573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33667/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 07:06:43.588238 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25641
2026-08-08T07:06:43Z chronyd exiting
I20260808 07:06:43.628321 18685 test_util.cc:175] -----------------------------------------------
I20260808 07:06:43.628405 18685 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786172779312146-18685-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 07:06:43.629153 18685 mini_oidc.cc:152] Starting JWKS server
I20260808 07:06:43.629889 18685 webserver.cc:460] Webserver started at https://127.0.0.1:33635/ using document root <none> and password file <none>
I20260808 07:06:43.629931 18685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:06:43.630057 18685 webserver.cc:460] Webserver started at http://127.0.0.1:40801/ using document root <none> and password file <none>
2026-08-08T07:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:43Z Disabled control of system clock
I20260808 07:06:43.639217 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38623
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:36079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:38623
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33635/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/testchainca.cert with env {}
W20260808 07:06:43.722018 25788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:43.722234 25788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:43.722256 25788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:43.723354 25788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:43.723390 25788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:43.723403 25788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:43.723416 25788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:43.723428 25788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:06:43.723440 25788 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:06:43.723456 25788 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33635/jwks.json
I20260808 07:06:43.724903 25788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:36079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:38623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33635/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:43.725117 25788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:43.725281 25788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:43.730232 25797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.730222 25801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.730301 25788 server_base.cc:1034] running on GCE node
W20260808 07:06:43.730240 25798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.730660 25788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:43.730942 25788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:43.732098 25788 hybrid_clock.cc:648] HybridClock initialized: now 1786172803732100 us; error 50 us; skew 500 ppm
I20260808 07:06:43.733377 25788 webserver.cc:460] Webserver started at http://127.18.63.126:34977/ using document root <none> and password file <none>
I20260808 07:06:43.733615 25788 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:43.733690 25788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:43.733841 25788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:43.734910 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "83a60c04a4144d32a4b63dc69b839edc"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "b4e37cc7c9454ac1ba3e90498e7b9c66"
server_key_iv: "078d53f073269b9f880867722e31d1b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.735291 25788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "83a60c04a4144d32a4b63dc69b839edc"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "b4e37cc7c9454ac1ba3e90498e7b9c66"
server_key_iv: "078d53f073269b9f880867722e31d1b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.736629 25788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:43.737303 25809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.737465 25788 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:43.737565 25788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "83a60c04a4144d32a4b63dc69b839edc"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "b4e37cc7c9454ac1ba3e90498e7b9c66"
server_key_iv: "078d53f073269b9f880867722e31d1b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.737653 25788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:43.770645 25788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:43.770992 25788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:43.771148 25788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:43.775353 25788 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:38623
I20260808 07:06:43.775393 25875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:38623 every 8 connection(s)
I20260808 07:06:43.775766 25788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:43.776494 25876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:43.779608 25876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc: Bootstrap starting.
I20260808 07:06:43.780200 25876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:43.780443 25876 log.cc:824] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:43.781307 25876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc: No bootstrap required, opened a new log
I20260808 07:06:43.782930 25876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } }
I20260808 07:06:43.783083 25876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:43.783107 25876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a60c04a4144d32a4b63dc69b839edc, State: Initialized, Role: FOLLOWER
I20260808 07:06:43.783246 25876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } }
I20260808 07:06:43.783327 25876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:43.783350 25876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:43.783383 25876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:43.783787 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25788
I20260808 07:06:43.783895 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:43.784137 18685 external_mini_cluster.cc:1428] Setting key 9ec956ede36f60eb9014ba63a451b64c
I20260808 07:06:43.784107 25876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } }
I20260808 07:06:43.784220 25876 leader_election.cc:302] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83a60c04a4144d32a4b63dc69b839edc; no voters:
I20260808 07:06:43.784443 25876 leader_election.cc:288] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:43.784464 25881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:43.784593 25881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [term 1 LEADER]: Becoming Leader. State: Replica: 83a60c04a4144d32a4b63dc69b839edc, State: Running, Role: LEADER
I20260808 07:06:43.784672 25881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } }
I20260808 07:06:43.784730 25876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:43.784993 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83a60c04a4144d32a4b63dc69b839edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } } }
I20260808 07:06:43.785068 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:43.785194 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83a60c04a4144d32a4b63dc69b839edc. Latest consensus state: current_term: 1 leader_uuid: "83a60c04a4144d32a4b63dc69b839edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a60c04a4144d32a4b63dc69b839edc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38623 } } }
I20260808 07:06:43.785264 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:43.785832 25888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:43.786429 25888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:43.788805 25888 catalog_manager.cc:1347] Generated new cluster ID: 4355122a9e624490992fd1ac5c0b207b
I20260808 07:06:43.788879 25888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:43.790740 25886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.784624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.18.63.126:38623)
0808 07:06:43.784812 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.784822 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.785169 (+ 347us) server_negotiation.cc:407] Connection header received
0808 07:06:43.785400 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.785409 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.785499 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.785646 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:43.786024 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.787633 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.788440 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.789052 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.789173 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.789923 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:43.789927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:43.789927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:43.790135 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:43.790180 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:43.790195 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:43.790268 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:43.790327 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 07:06:43.790514 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260808 07:06:43.791672 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:43.806263 25888 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:43.806787 25888 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:43.811539 25888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83a60c04a4144d32a4b63dc69b839edc: Generated new TSK 0
I20260808 07:06:43.811730 25888 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 07:06:43.834410 25918 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 07:06:43.834501 25918 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 07:06:43.834861 25886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:43.820499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.18.63.126:38623)
0808 07:06:43.820558 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:43.820561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:43.820665 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:06:43.820733 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:43.820733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:43.820746 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:43.820822 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:06:43.820925 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.821778 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.822513 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.822848 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.822912 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.823063 (+ 151us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:06:43.834632 (+ 11569us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:06:43.834787 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 07:06:43.834920 25886 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 07:06:43.834932 25917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:43.820425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38623 (local address 127.0.0.1:55148)
0808 07:06:43.820591 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:43.820601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:43.820665 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:43.820818 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:43.820819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:43.820821 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:06:43.820870 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:43.820872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.821851 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.821851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:43.822434 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:43.822435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:43.822931 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:43.822932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:43.822974 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:43.822978 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:06:43.834740 (+ 11762us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 07:06:43.834841 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.63.126:38623: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260808 07:06:43.835242 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25788
2026-08-08T07:06:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 07:06:43.883512 18685 mini_oidc.cc:152] Starting JWKS server
I20260808 07:06:43.884178 18685 webserver.cc:460] Webserver started at https://127.0.0.1:44141/ using document root <none> and password file <none>
I20260808 07:06:43.884218 18685 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 07:06:43.884357 18685 webserver.cc:460] Webserver started at http://127.0.0.1:34659/ using document root <none> and password file <none>
2026-08-08T07:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:43Z Disabled control of system clock
I20260808 07:06:43.893138 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43855
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43855
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44141/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:06:43.975040 25925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:43.975242 25925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:43.975262 25925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:43.976425 25925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:43.976473 25925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:43.976486 25925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:43.976500 25925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:43.976511 25925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:06:43.976591 25925 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:06:43.976608 25925 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44141/jwks.json
I20260808 07:06:43.978039 25925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44141/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:43.978312 25925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:43.978605 25925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:43.983711 25934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.983701 25931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:43.983707 25930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:43.984004 25925 server_base.cc:1034] running on GCE node
I20260808 07:06:43.984300 25925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:43.984586 25925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:43.985742 25925 hybrid_clock.cc:648] HybridClock initialized: now 1786172803985717 us; error 54 us; skew 500 ppm
I20260808 07:06:43.986958 25925 webserver.cc:460] Webserver started at http://127.18.63.126:35627/ using document root <none> and password file <none>
I20260808 07:06:43.987246 25925 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:43.987318 25925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:43.987458 25925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:43.988547 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "43c358f794e34d449d5d71965df77fd0"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "50fa5ddce339fbe24612af481d024d8e"
server_key_iv: "7607b71a469dbd4dd978a204a88cde31"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.988912 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "43c358f794e34d449d5d71965df77fd0"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "50fa5ddce339fbe24612af481d024d8e"
server_key_iv: "7607b71a469dbd4dd978a204a88cde31"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.990322 25925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:43.991021 25942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:43.991204 25925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:43.991305 25925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "43c358f794e34d449d5d71965df77fd0"
format_stamp: "Formatted at 2026-08-08 07:06:43 on dist-test-slave-bw88"
server_key: "50fa5ddce339fbe24612af481d024d8e"
server_key_iv: "7607b71a469dbd4dd978a204a88cde31"
server_key_version: "encryptionkey@0"
I20260808 07:06:43.991391 25925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:44.014361 25925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:44.014721 25925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:44.014878 25925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:44.018994 25925 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43855
I20260808 07:06:44.019050 26012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43855 every 8 connection(s)
I20260808 07:06:44.019400 25925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:44.020079 26013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:44.022946 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0: Bootstrap starting.
I20260808 07:06:44.023556 26013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:44.023857 26013 log.cc:824] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:44.024750 26013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0: No bootstrap required, opened a new log
I20260808 07:06:44.026451 26013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } }
I20260808 07:06:44.026593 26013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:44.026616 26013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c358f794e34d449d5d71965df77fd0, State: Initialized, Role: FOLLOWER
I20260808 07:06:44.026806 26013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } }
I20260808 07:06:44.026891 26013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:44.026942 26013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:44.026993 26013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:44.027539 26013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } }
I20260808 07:06:44.027658 26013 leader_election.cc:302] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c358f794e34d449d5d71965df77fd0; no voters:
I20260808 07:06:44.027834 26013 leader_election.cc:288] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:44.027892 26017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:44.027891 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 25925
I20260808 07:06:44.028003 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:44.028083 26017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 43c358f794e34d449d5d71965df77fd0, State: Running, Role: LEADER
I20260808 07:06:44.028112 26013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:44.028214 18685 external_mini_cluster.cc:1428] Setting key 7ad077f6c913d1c86c388562372867a4
I20260808 07:06:44.028196 26017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } }
I20260808 07:06:44.028563 26017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c358f794e34d449d5d71965df77fd0. Latest consensus state: current_term: 1 leader_uuid: "43c358f794e34d449d5d71965df77fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } } }
I20260808 07:06:44.028581 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c358f794e34d449d5d71965df77fd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c358f794e34d449d5d71965df77fd0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43855 } } }
I20260808 07:06:44.028664 26017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:44.028669 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:44.029357 26029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:44.029955 26029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:44.031594 26029 catalog_manager.cc:1347] Generated new cluster ID: 097614e568a941e7b6194d10f6298d01
I20260808 07:06:44.031649 26029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:44.034909 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:44.028894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.18.63.126:43855)
0808 07:06:44.029151 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:44.029160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:44.029194 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:06:44.029321 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:44.029330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:44.029448 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:44.029888 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:44.029988 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.031426 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.032569 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.033192 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.033316 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.034069 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:44.034073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:44.034074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:44.034295 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:44.034337 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:44.034351 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:44.034418 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:44.034491 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:06:44.034689 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
I20260808 07:06:44.035821 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:44.044893 26029 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:44.045456 26029 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:44.047215 26029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c358f794e34d449d5d71965df77fd0: Generated new TSK 0
I20260808 07:06:44.047469 26029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:44.062294 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:44.058498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.18.63.126:43855)
0808 07:06:44.058553 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:44.058555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:44.058694 (+ 139us) server_negotiation.cc:407] Connection header received
0808 07:06:44.058782 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:44.058783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:44.058793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:44.058852 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:06:44.059021 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.059503 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.060642 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.060836 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.060886 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.061097 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:06:44.062092 (+ 995us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44141/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:06:44.062228 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44141/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 07:06:44.062352 26032 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44141/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 07:06:44.062422 26052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:44.058384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43855 (local address 127.0.0.1:60984)
0808 07:06:44.058612 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:44.058625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:44.058704 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:44.058871 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:44.058872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:44.058874 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:06:44.058953 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:44.058955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.059586 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.059587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:44.060551 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:44.060554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.060921 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.060922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:44.060987 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.060994 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:06:44.062200 (+ 1206us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44141/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 07:06:44.062316 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.63.126:43855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44141/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260808 07:06:44.062595 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 25925
E20260808 07:06:44.082954 26053 webserver.cc:533] Webserver: sslize failed:
E20260808 07:06:44.083071 26053 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T07:06:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T07:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:44Z Disabled control of system clock
I20260808 07:06:44.123824 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:35145
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:35145
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/testchainca.cert with env {}
W20260808 07:06:44.206561 26058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:44.206766 26058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:44.206790 26058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:44.207948 26058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:44.207988 26058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:44.208000 26058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:44.208014 26058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:44.208024 26058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 07:06:44.208042 26058 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 07:06:44.208056 26058 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 07:06:44.209467 26058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:35145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:35145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:44.209652 26058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:44.209800 26058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:44.214574 26068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:44.214581 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:44.214605 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:44.214802 26058 server_base.cc:1034] running on GCE node
I20260808 07:06:44.215034 26058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:44.215315 26058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:44.216468 26058 hybrid_clock.cc:648] HybridClock initialized: now 1786172804216445 us; error 55 us; skew 500 ppm
I20260808 07:06:44.217721 26058 webserver.cc:460] Webserver started at http://127.18.63.126:38689/ using document root <none> and password file <none>
I20260808 07:06:44.217965 26058 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:44.218034 26058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:44.218189 26058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:44.219205 26058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "72a49d92772148948366a49079af660d"
format_stamp: "Formatted at 2026-08-08 07:06:44 on dist-test-slave-bw88"
server_key: "411596e8bbc1b355f7455dfbfb1f59e5"
server_key_iv: "d5d6ebccbb509f337276ca47426cbd0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:44.219559 26058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "72a49d92772148948366a49079af660d"
format_stamp: "Formatted at 2026-08-08 07:06:44 on dist-test-slave-bw88"
server_key: "411596e8bbc1b355f7455dfbfb1f59e5"
server_key_iv: "d5d6ebccbb509f337276ca47426cbd0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:44.220781 26058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:44.221511 26076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:44.221683 26058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:44.221781 26058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "72a49d92772148948366a49079af660d"
format_stamp: "Formatted at 2026-08-08 07:06:44 on dist-test-slave-bw88"
server_key: "411596e8bbc1b355f7455dfbfb1f59e5"
server_key_iv: "d5d6ebccbb509f337276ca47426cbd0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:44.221861 26058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:44.246992 26058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:44.247362 26058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:44.247539 26058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:44.251535 26058 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:35145
I20260808 07:06:44.251667 26148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:35145 every 8 connection(s)
I20260808 07:06:44.251963 26058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:44.253011 26149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:44.255403 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d: Bootstrap starting.
I20260808 07:06:44.255987 26149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:44.256237 26149 log.cc:824] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:44.257016 26149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d: No bootstrap required, opened a new log
I20260808 07:06:44.258667 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26058
I20260808 07:06:44.258775 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:44.258836 26149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } }
I20260808 07:06:44.258982 18685 external_mini_cluster.cc:1428] Setting key 6b3fbcc291eb997fdd6f77d1d13573cf
I20260808 07:06:44.259022 26149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:44.259069 26149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a49d92772148948366a49079af660d, State: Initialized, Role: FOLLOWER
I20260808 07:06:44.259225 26149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } }
I20260808 07:06:44.259326 26149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:44.259372 26149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:44.259428 26149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:44.260128 26149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } }
I20260808 07:06:44.260263 26149 leader_election.cc:302] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a49d92772148948366a49079af660d; no voters:
I20260808 07:06:44.260459 26149 leader_election.cc:288] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:44.260493 26155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:44.260663 26155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [term 1 LEADER]: Becoming Leader. State: Replica: 72a49d92772148948366a49079af660d, State: Running, Role: LEADER
I20260808 07:06:44.260749 26155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } }
I20260808 07:06:44.260774 26149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:44.261054 26155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a49d92772148948366a49079af660d. Latest consensus state: current_term: 1 leader_uuid: "72a49d92772148948366a49079af660d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } } }
I20260808 07:06:44.261118 26155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:44.261232 26156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a49d92772148948366a49079af660d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a49d92772148948366a49079af660d" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 35145 } } }
I20260808 07:06:44.261319 26156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:44.261384 26160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:44.261924 26160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:44.263533 26160 catalog_manager.cc:1347] Generated new cluster ID: 3ae9d4fc844b4bff814fcd0e1387021e
I20260808 07:06:44.263593 26160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:44.265610 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:44.259618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.18.63.126:35145)
0808 07:06:44.259845 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:44.259852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:44.259877 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:06:44.259974 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:44.259979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:44.260043 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:44.260216 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:44.260458 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.262349 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.263533 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.264105 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.264235 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.264905 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:44.264909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:44.264909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:06:44.265119 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:44.265163 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:06:44.265176 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:44.265234 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:44.265290 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:06:44.265393 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260808 07:06:44.266587 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:06:44.276598 26160 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:44.277302 26160 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:44.282684 26160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a49d92772148948366a49079af660d: Generated new TSK 0
I20260808 07:06:44.282855 26160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:44.330109 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:44.290550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.18.63.126:35145)
0808 07:06:44.290617 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:44.290620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:44.290765 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:06:44.290845 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:44.290846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:44.290866 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:44.290926 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0808 07:06:44.291070 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.291554 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.292721 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.293225 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.293280 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.293496 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 07:06:44.329887 (+ 36391us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:06:44.330050 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34678: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 07:06:44.330201 26154 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34678: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 07:06:44.330307 26187 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:44.290502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:35145 (local address 127.0.0.1:34678)
0808 07:06:44.290693 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:44.290705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:44.290775 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:44.290941 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:44.290942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:44.290944 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 07:06:44.291009 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:44.291011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.291650 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.291651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:44.292642 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:44.292645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:44.293308 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:44.293309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:44.293395 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:44.293402 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 07:06:44.330042 (+ 36640us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 07:06:44.330196 (+ 154us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.63.126:35145: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":115,"threads_started":1}
I20260808 07:06:44.330721 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26058
2026-08-08T07:06:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (237 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:44 dist-test-slave-bw88 krb5kdc[26202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:44 dist-test-slave-bw88 krb5kdc[26202](info): set up 2 sockets
Aug 08 07:06:44 dist-test-slave-bw88 krb5kdc[26202](info): commencing operation
krb5kdc: starting...
W20260808 07:06:45.709507 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:45 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:45.833702 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42815
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:45.917418 26217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:45.917626 26217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:45.917649 26217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:45.918871 26217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:45.918912 26217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:45.918926 26217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:45.918977 26217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:45.919003 26217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:45.920513 26217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:45.920760 26217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:45.920981 26217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:45.926059 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:45.926123 26226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:45.926165 26217 server_base.cc:1034] running on GCE node
W20260808 07:06:45.926335 26230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:45.926601 26217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:45.926852 26217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:45.927996 26217 hybrid_clock.cc:648] HybridClock initialized: now 1786172805927983 us; error 35 us; skew 500 ppm
Aug 08 07:06:45 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172805, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:45.930740 26217 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:45.931298 26217 webserver.cc:460] Webserver started at http://127.18.63.126:32783/ using document root <none> and password file <none>
I20260808 07:06:45.931555 26217 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:45.931603 26217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:45.931749 26217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:45.932786 26217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "e11d4eac6d2644809c62d672606dffa0"
format_stamp: "Formatted at 2026-08-08 07:06:45 on dist-test-slave-bw88"
server_key: "5e47e4dbdaf02ed8304f4d351b4c2d1b"
server_key_iv: "dacf4177b4078435d0fb57394f88ca4e"
server_key_version: "encryptionkey@0"
I20260808 07:06:45.933190 26217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "e11d4eac6d2644809c62d672606dffa0"
format_stamp: "Formatted at 2026-08-08 07:06:45 on dist-test-slave-bw88"
server_key: "5e47e4dbdaf02ed8304f4d351b4c2d1b"
server_key_iv: "dacf4177b4078435d0fb57394f88ca4e"
server_key_version: "encryptionkey@0"
I20260808 07:06:45.934532 26217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:45.935315 26239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:45.935532 26217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:45.935618 26217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "e11d4eac6d2644809c62d672606dffa0"
format_stamp: "Formatted at 2026-08-08 07:06:45 on dist-test-slave-bw88"
server_key: "5e47e4dbdaf02ed8304f4d351b4c2d1b"
server_key_iv: "dacf4177b4078435d0fb57394f88ca4e"
server_key_version: "encryptionkey@0"
I20260808 07:06:45.935698 26217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:45.954507 26217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:45.956820 26217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:45.956985 26217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:45.961218 26217 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:42815
I20260808 07:06:45.961261 26303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:42815 every 8 connection(s)
I20260808 07:06:45.961670 26217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:45.962594 26305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:45.964838 26305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0: Bootstrap starting.
I20260808 07:06:45.965461 26305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:45.965716 26305 log.cc:824] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:45.966393 26305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0: No bootstrap required, opened a new log
I20260808 07:06:45.967926 26305 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } }
I20260808 07:06:45.968075 26305 raft_consensus.cc:374] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:45.968098 26305 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11d4eac6d2644809c62d672606dffa0, State: Initialized, Role: FOLLOWER
I20260808 07:06:45.968240 26305 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } }
I20260808 07:06:45.968319 26305 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:45.968341 26305 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:45.968386 26305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:45.968571 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26217
I20260808 07:06:45.968675 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:45.968865 18685 external_mini_cluster.cc:1428] Setting key 746dcef1f0da04f21a65671f31660731
I20260808 07:06:45.969099 26305 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } }
I20260808 07:06:45.969215 26305 leader_election.cc:302] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11d4eac6d2644809c62d672606dffa0; no voters:
I20260808 07:06:45.969449 26305 leader_election.cc:288] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:45.969726 26305 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:45.969753 26311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:45.969877 26311 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [term 1 LEADER]: Becoming Leader. State: Replica: e11d4eac6d2644809c62d672606dffa0, State: Running, Role: LEADER
I20260808 07:06:45.969978 26311 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } }
I20260808 07:06:45.970345 26311 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11d4eac6d2644809c62d672606dffa0. Latest consensus state: current_term: 1 leader_uuid: "e11d4eac6d2644809c62d672606dffa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } } }
I20260808 07:06:45.970436 26311 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:45.970381 26315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e11d4eac6d2644809c62d672606dffa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11d4eac6d2644809c62d672606dffa0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42815 } } }
I20260808 07:06:45.970613 26315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:45.970772 26324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:45.971319 26324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:45.972941 26324 catalog_manager.cc:1347] Generated new cluster ID: 97272ed777cf4d96ad326ee3ec57d1fd
I20260808 07:06:45.972992 26324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:45 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:45.981593 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:45.969400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.18.63.126:42815)
0808 07:06:45.969709 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:45.969719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:45.969757 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:06:45.969919 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:45.969927 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:45.969986 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:45.971075 (+ 1089us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:45.971172 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:45.972632 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:45.973707 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:45.974302 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:45.974397 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:45.977220 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:45.977225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:45.977225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:45.977240 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:45.979882 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:45.980270 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:45.980274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:45.980274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:45.980388 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:45.980604 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:45.980605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:45.980605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:45.980931 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:45.981115 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:45.981200 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:06:45.981367 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":84,"threads_started":1}
I20260808 07:06:45.984633 26324 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:45.985246 26324 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:45.993144 26324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e11d4eac6d2644809c62d672606dffa0: Generated new TSK 0
I20260808 07:06:45.993371 26324 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:46.026180 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:46.110641 26340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:46.110863 26340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:46.110890 26340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:46.112244 26340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:46.112293 26340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:46.112323 26340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:46.114060 26340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:46.114305 26340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:46.114542 26340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:46.120023 26347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:46.120064 26340 server_base.cc:1034] running on GCE node
W20260808 07:06:46.119972 26348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:46.119972 26353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:46.120348 26340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:46.120600 26340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:46.121755 26340 hybrid_clock.cc:648] HybridClock initialized: now 1786172806121749 us; error 35 us; skew 500 ppm
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.124289 26340 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:46.124709 26340 webserver.cc:460] Webserver started at http://127.18.63.65:40581/ using document root <none> and password file <none>
I20260808 07:06:46.124948 26340 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:46.125013 26340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:46.125181 26340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:46.126278 26340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "7d06cb37c5a349e3a23e41a8e404484e"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "951afc3c64c5d61523ebbc57990889fb"
server_key_iv: "a49b788155e2d2c6c4b3c76137a533ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.126657 26340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "7d06cb37c5a349e3a23e41a8e404484e"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "951afc3c64c5d61523ebbc57990889fb"
server_key_iv: "a49b788155e2d2c6c4b3c76137a533ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.127923 26340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:46.128620 26362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.128794 26340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:46.128890 26340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "7d06cb37c5a349e3a23e41a8e404484e"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "951afc3c64c5d61523ebbc57990889fb"
server_key_iv: "a49b788155e2d2c6c4b3c76137a533ff"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.128969 26340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:46.138093 26340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:46.140296 26340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:46.140446 26340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:46.140674 26340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:46.140974 26340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:46.141026 26340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.141064 26340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:46.141093 26340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.147162 26340 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:38737
I20260808 07:06:46.147234 26523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:38737 every 8 connection(s)
I20260808 07:06:46.147540 26340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:46.151713 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26340
I20260808 07:06:46.151823 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:46.152047 18685 external_mini_cluster.cc:1428] Setting key bf30d6164eeffc3f09c1967db322a3d1
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:46.157483 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.148322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:40269 (local address 127.18.63.126:42815)
0808 07:06:46.148377 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.148379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.148551 (+ 172us) server_negotiation.cc:407] Connection header received
0808 07:06:46.149473 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.149474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.149484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.149559 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.150125 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.150633 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.151613 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.151942 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.152023 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.154649 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.154651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.154651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.154658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.156078 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.156466 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.156468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.156468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.156534 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.156808 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.156809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.156809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.156927 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.157004 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.157276 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.157389 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:06:46.157675 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.148280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42815 (local address 127.18.63.65:40269)
0808 07:06:46.148433 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:46.148473 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:46.149380 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:46.149612 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:46.149615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:46.149856 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:46.150039 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.150044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.150714 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.150714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.151508 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.151510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.152029 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.152030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.152179 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.152690 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:46.152700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:46.154545 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:46.156154 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.156161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.156162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.156377 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.156623 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.156624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.156624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.156731 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.156995 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:46.157003 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:46.157143 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 07:06:46.157251 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 07:06:46.157434 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260808 07:06:46.158229 26524 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42815
I20260808 07:06:46.158342 26524 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:46.158516 26524 heartbeater.cc:511] Master 127.18.63.126:42815 requested a full tablet report, sending...
I20260808 07:06:46.159102 26259 ts_manager.cc:194] Registered new tserver with Master: 7d06cb37c5a349e3a23e41a8e404484e (127.18.63.65:38737)
I20260808 07:06:46.160073 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:40269
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:46.180720 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:46.264330 26529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:46.264531 26529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:46.264555 26529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:46.265712 26529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:46.265751 26529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:46.265777 26529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:46.267164 26529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:46.267356 26529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:46.267530 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:46.272799 26538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:46.272840 26535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:46.272822 26534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:46.272986 26529 server_base.cc:1034] running on GCE node
I20260808 07:06:46.273196 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:46.273475 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:46.274642 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786172806274632 us; error 34 us; skew 500 ppm
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.277210 26529 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:46.277666 26529 webserver.cc:460] Webserver started at http://127.18.63.66:38431/ using document root <none> and password file <none>
I20260808 07:06:46.277880 26529 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:46.277932 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:46.278025 26529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:46.279026 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "180e5ad32fdd4d239978ad73a38979a1"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "0e64c6d5c618614d2609d8309c0896c4"
server_key_iv: "5b85f650981885c005bf644a7bf47a60"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.279337 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "180e5ad32fdd4d239978ad73a38979a1"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "0e64c6d5c618614d2609d8309c0896c4"
server_key_iv: "5b85f650981885c005bf644a7bf47a60"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.280629 26529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:46.281394 26549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.281546 26529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:46.281642 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "180e5ad32fdd4d239978ad73a38979a1"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "0e64c6d5c618614d2609d8309c0896c4"
server_key_iv: "5b85f650981885c005bf644a7bf47a60"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.281700 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:46.302376 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:46.304284 26529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:46.304400 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:46.304602 26529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:46.304939 26529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:46.304975 26529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.305017 26529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:46.305044 26529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.311066 26529 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:32913
I20260808 07:06:46.311156 26688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:32913 every 8 connection(s)
I20260808 07:06:46.311468 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:46.316219 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26529
I20260808 07:06:46.316354 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:46.316567 18685 external_mini_cluster.cc:1428] Setting key 244eecffec324b670c23f21ab622bcee
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:46.321448 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.312287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:35657 (local address 127.18.63.126:42815)
0808 07:06:46.312360 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.312362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.312583 (+ 221us) server_negotiation.cc:407] Connection header received
0808 07:06:46.313543 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.313545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.313555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.313616 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.314205 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.314718 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.315700 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.315892 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.315944 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.318327 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.318328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.318329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.318337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.319895 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.320411 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.320411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.320412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.320463 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.320745 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.320745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.320745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.320845 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.320893 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.321238 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.321314 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:06:46.321708 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.312250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42815 (local address 127.18.63.66:35657)
0808 07:06:46.312428 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:46.312472 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:46.313446 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:46.313673 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:46.313676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:46.313927 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:46.314131 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.314136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.314799 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.314800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.315628 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.315630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.315964 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.315981 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.316095 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.316706 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:46.316715 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:46.318241 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:46.319985 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.319993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.319996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.320332 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.320527 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.320528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.320529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.320674 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.320905 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:46.320911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:46.321092 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 07:06:46.321221 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 07:06:46.321427 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260808 07:06:46.322257 26689 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42815
I20260808 07:06:46.322369 26689 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:46.322571 26689 heartbeater.cc:511] Master 127.18.63.126:42815 requested a full tablet report, sending...
I20260808 07:06:46.323033 26259 ts_manager.cc:194] Registered new tserver with Master: 180e5ad32fdd4d239978ad73a38979a1 (127.18.63.66:32913)
I20260808 07:06:46.323414 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:35657
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:46.344307 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:46.431630 26696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:46.431901 26696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:46.431947 26696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:46.433470 26696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:46.433543 26696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:46.433593 26696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:46.435154 26696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:46.435642 26696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:46.435904 26696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:46.441183 26703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:46.441197 26705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:46.441287 26701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:46.441593 26696 server_base.cc:1034] running on GCE node
I20260808 07:06:46.441856 26696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:46.442070 26696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:46.443212 26696 hybrid_clock.cc:648] HybridClock initialized: now 1786172806443196 us; error 22 us; skew 500 ppm
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.446262 26696 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:46.446766 26696 webserver.cc:460] Webserver started at http://127.18.63.67:43929/ using document root <none> and password file <none>
I20260808 07:06:46.446998 26696 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:46.447046 26696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:46.447225 26696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:46.448324 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "64adabce2feb410eac7d27098ae1021d"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "736a2639ec5f8ded3689eee4ed76f233"
server_key_iv: "fa48259f53ee4ba06d56f7dacf37180e"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.448819 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "64adabce2feb410eac7d27098ae1021d"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "736a2639ec5f8ded3689eee4ed76f233"
server_key_iv: "fa48259f53ee4ba06d56f7dacf37180e"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.450724 26696 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 07:06:46.451591 26713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.451836 26696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:46.451958 26696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "64adabce2feb410eac7d27098ae1021d"
format_stamp: "Formatted at 2026-08-08 07:06:46 on dist-test-slave-bw88"
server_key: "736a2639ec5f8ded3689eee4ed76f233"
server_key_iv: "fa48259f53ee4ba06d56f7dacf37180e"
server_key_version: "encryptionkey@0"
I20260808 07:06:46.452059 26696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:46.474514 26696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:46.477017 26696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:46.477195 26696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:46.477530 26696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:46.477869 26696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:46.477928 26696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.477978 26696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:46.478003 26696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:46.484184 26696 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:35925
I20260808 07:06:46.484226 26854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:35925 every 8 connection(s)
I20260808 07:06:46.484620 26696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:46.489850 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26696
I20260808 07:06:46.489961 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:46.490178 18685 external_mini_cluster.cc:1428] Setting key 59400c13c675a7c71ca3c4cec75cd819
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:46.494632 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.485532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:49197 (local address 127.18.63.126:42815)
0808 07:06:46.485578 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.485580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.485816 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:06:46.486824 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.486826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.486841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.486899 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.487560 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.488073 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.489130 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.489378 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.489445 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.491922 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.491924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.491925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.491934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.493151 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.493630 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.493632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.493632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.493700 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.493967 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.493968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.493969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.494087 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.494175 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.494455 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.494541 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:06:46.494838 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.485537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42815 (local address 127.18.63.67:49197)
0808 07:06:46.485694 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:46.485731 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:46.486700 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:46.486969 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:46.486973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:46.487250 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:46.487451 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.487456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.488187 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.488188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.489024 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:46.489027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.489477 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.489478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:46.489602 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.490224 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:46.490237 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:46.491824 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:46.493251 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.493258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.493260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.493544 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.493773 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:46.493774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:46.493774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:46.493891 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:46.494163 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:46.494170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:46.494315 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 07:06:46.494431 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 07:06:46.494593 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260808 07:06:46.495317 26855 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42815
I20260808 07:06:46.495398 26855 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:46.495636 26855 heartbeater.cc:511] Master 127.18.63.126:42815 requested a full tablet report, sending...
I20260808 07:06:46.496094 26259 ts_manager.cc:194] Registered new tserver with Master: 64adabce2feb410eac7d27098ae1021d (127.18.63.67:35925)
I20260808 07:06:46.496510 26259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:49197
I20260808 07:06:46.502068 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/test-admin.keytab.
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.518200 18685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 07:06:46.520052 18685 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc' of type FILE
W20260808 07:06:46.520056 26862 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:46.519121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.18.63.65:38737)
0808 07:06:46.519309 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.519314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.519326 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:46.519355 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.519357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.519371 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.519449 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.519930 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44340: BlockingRecv error: recv got EOF from 127.0.0.1:44340 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.521055 18685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.522010 18685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:46.530272 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.522520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.18.63.65:38737)
0808 07:06:46.522566 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.522567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.522640 (+ 73us) server_negotiation.cc:407] Connection header received
0808 07:06:46.522721 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.522722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.522741 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.522806 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.523236 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.523995 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.524758 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.525101 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.525187 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.527478 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.527486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.527486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.527492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.528977 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.529495 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.529498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.529499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.529556 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.529793 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.529794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.529794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.529933 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.530037 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.530124 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.530191 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:06:46.532577 26862 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:46.531865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.18.63.65:38737)
0808 07:06:46.531934 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.531936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.532024 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:06:46.532102 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.532103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.532117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.532187 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.532539 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44356: BlockingRecv error: recv got EOF from 127.0.0.1:44356 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 07:06:46.532738 18685 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc' of type FILE
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.533914 18685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.534861 18685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:46.542348 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.535418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.18.63.65:38737)
0808 07:06:46.535492 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.535495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.535589 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:06:46.535698 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.535699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.535711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.535760 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.536148 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.536667 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.537487 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.537732 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.537817 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.540032 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.540034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.540034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.540040 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.541305 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.541674 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.541675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.541675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.541727 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.541897 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.541897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.541898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.541989 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.542059 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.542182 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.542285 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 07:06:46.543936 26862 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:46.543351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.18.63.65:38737)
0808 07:06:46.543402 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.543404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.543445 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:06:46.543518 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.543519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.543529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.543622 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.543888 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44364: BlockingRecv error: recv got EOF from 127.0.0.1:44364 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 07:06:46.544137 18685 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786172779312146-18685-0/krb5kdc/krb5cc)
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.545138 18685 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:46.546066 18685 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:06:46.553216 26862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:46.546551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.18.63.65:38737)
0808 07:06:46.546628 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:46.546630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:46.546720 (+ 90us) server_negotiation.cc:407] Connection header received
0808 07:06:46.546793 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:46.546793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:46.546803 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:46.546870 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:46.547249 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.547711 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.548399 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:46.548591 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:46.548664 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:46.550726 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:46.550728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:46.550728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:46.550735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:46.552110 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.552454 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.552455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.552455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.552513 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:46.552714 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:46.552715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:46.552716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:46.552818 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:46.552918 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:46.553028 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:06:46.553113 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:06:46.553892 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26340
I20260808 07:06:46.566728 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26529
I20260808 07:06:46.578852 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26696
I20260808 07:06:46.589448 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26217
2026-08-08T07:06:46Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2072 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26876](info): set up 2 sockets
Aug 08 07:06:46 dist-test-slave-bw88 krb5kdc[26876](info): commencing operation
krb5kdc: starting...
W20260808 07:06:47.781082 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:47 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab.
Entry for principal oryx/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab.
I20260808 07:06:47.905979 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42871
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab
--principal=oryx/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:47.989153 26891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:47.989382 26891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:47.989405 26891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:47.990605 26891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:47.990656 26891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:47.990671 26891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:47.990684 26891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:47.990696 26891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:47.992247 26891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab
--principal=oryx/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:47.992515 26891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:47.992767 26891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:47.997817 26900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:47.997808 26898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:47.997978 26891 server_base.cc:1034] running on GCE node
W20260808 07:06:47.997879 26897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:47.998270 26891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:47.998564 26891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:47.999739 26891 hybrid_clock.cc:648] HybridClock initialized: now 1786172807999723 us; error 48 us; skew 500 ppm
Aug 08 07:06:47 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.005208 26891 init.cc:377] Logged in from keytab as oryx/127.18.63.126@KRBTEST.COM (short username oryx)
I20260808 07:06:48.005713 26891 webserver.cc:460] Webserver started at http://127.18.63.126:43129/ using document root <none> and password file <none>
I20260808 07:06:48.005980 26891 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.006052 26891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.006212 26891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:48.007314 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "928f91425d204020b989476350668cf6"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "19e79ef1e00612b96aee7263c2e474c5"
server_key_iv: "ce6baabf82a9ad204c3e504c79084d30"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.007692 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "928f91425d204020b989476350668cf6"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "19e79ef1e00612b96aee7263c2e474c5"
server_key_iv: "ce6baabf82a9ad204c3e504c79084d30"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.009054 26891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:06:48.009883 26913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.010066 26891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.010164 26891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "928f91425d204020b989476350668cf6"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "19e79ef1e00612b96aee7263c2e474c5"
server_key_iv: "ce6baabf82a9ad204c3e504c79084d30"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.010252 26891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.030224 26891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.032735 26891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:48.032915 26891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.037393 26891 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:42871
I20260808 07:06:48.037428 26984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:42871 every 8 connection(s)
I20260808 07:06:48.037810 26891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:48.038861 26985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:48.040974 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 26891
I20260808 07:06:48.041098 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:48.041320 18685 external_mini_cluster.cc:1428] Setting key 33cdb4dbca2c389340c45849e8ce5eef
I20260808 07:06:48.041380 26985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Bootstrap starting.
I20260808 07:06:48.042145 26985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:48.042454 26985 log.cc:824] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:48.043192 26985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: No bootstrap required, opened a new log
I20260808 07:06:48.044997 26985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } }
I20260808 07:06:48.045156 26985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:48.045202 26985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928f91425d204020b989476350668cf6, State: Initialized, Role: FOLLOWER
I20260808 07:06:48.045360 26985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } }
I20260808 07:06:48.045475 26985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:48.045521 26985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:48.045571 26985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:48.046092 26985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } }
I20260808 07:06:48.046219 26985 leader_election.cc:302] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 928f91425d204020b989476350668cf6; no voters:
I20260808 07:06:48.046384 26985 leader_election.cc:288] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:48.046422 26990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:48.046653 26990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 LEADER]: Becoming Leader. State: Replica: 928f91425d204020b989476350668cf6, State: Running, Role: LEADER
I20260808 07:06:48.046680 26985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:48.046758 26990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } }
I20260808 07:06:48.047117 26990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928f91425d204020b989476350668cf6. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } }
I20260808 07:06:48.047127 26991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } }
I20260808 07:06:48.047207 26990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:48.047252 26991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
W20260808 07:06:48.048552 27015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:48.048750 27015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:48.048812 27004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:48.048967 27004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:48.052695 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.041944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.18.63.126:42871)
0808 07:06:48.042134 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.042143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.042177 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:06:48.042721 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.042728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.042777 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.042923 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.043654 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.044517 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.045376 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.045897 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.046013 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.048958 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.048962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.048963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.048976 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.051087 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.051439 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.051442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.051443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.051547 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.051762 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.051763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.051764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.052086 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.052250 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.052320 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.052468 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260808 07:06:48.056854 27004 catalog_manager.cc:1347] Generated new cluster ID: 47124f5937ad452c84cf1f8f5c4f010f
I20260808 07:06:48.056910 27004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:48.071565 27004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:48.072110 27004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:48.078745 27004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Generated new TSK 0
I20260808 07:06:48.078938 27004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
I20260808 07:06:48.117980 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:48.248070 27022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:48.248287 27022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:48.248313 27022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:48.249594 27022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:48.249639 27022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:48.249668 27022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:48.251302 27022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:48.251577 27022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:48.251852 27022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:48.257149 27030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.257157 27029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.257212 27032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.257541 27022 server_base.cc:1034] running on GCE node
I20260808 07:06:48.257797 27022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:48.258050 27022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:48.259217 27022 hybrid_clock.cc:648] HybridClock initialized: now 1786172808259199 us; error 30 us; skew 500 ppm
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.261919 27022 init.cc:377] Logged in from keytab as oryx/127.18.63.65@KRBTEST.COM (short username oryx)
I20260808 07:06:48.262379 27022 webserver.cc:460] Webserver started at http://127.18.63.65:38275/ using document root <none> and password file <none>
I20260808 07:06:48.262598 27022 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.262640 27022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.262730 27022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:48.263738 27022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "42095ed2821f4ddd9b3281ff509327a1"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c559d18bbdb7c15364f8739c44c07b1e"
server_key_iv: "d7991a97c7e36a256899a91fae4a3907"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.264139 27022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "42095ed2821f4ddd9b3281ff509327a1"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c559d18bbdb7c15364f8739c44c07b1e"
server_key_iv: "d7991a97c7e36a256899a91fae4a3907"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.265576 27022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:48.266403 27040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.266638 27022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:48.266729 27022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "42095ed2821f4ddd9b3281ff509327a1"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c559d18bbdb7c15364f8739c44c07b1e"
server_key_iv: "d7991a97c7e36a256899a91fae4a3907"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.266785 27022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.281481 27022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.283875 27022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:48.284001 27022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.284192 27022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:48.284512 27022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:48.284546 27022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.284569 27022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:48.284581 27022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.290942 27022 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46189
I20260808 07:06:48.291043 27192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46189 every 8 connection(s)
I20260808 07:06:48.291347 27022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:48.294579 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27022
I20260808 07:06:48.294693 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:48.294901 18685 external_mini_cluster.cc:1428] Setting key ef73fba1979deb794ed259b66eea5134
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:48.301908 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.292138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:41819)
0808 07:06:48.292323 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.292365 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.292879 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.293092 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.293095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.293777 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.293973 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.293979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.294836 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.294837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.295863 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.295866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.296366 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.296367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.296468 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.297024 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.297032 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.298646 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.300486 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.300495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.300497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.300736 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.300958 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.300958 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.300959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.301057 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.301401 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.301407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.301530 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.301616 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.301740 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260808 07:06:48.301923 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.292250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:41819 (local address 127.18.63.126:42871)
0808 07:06:48.292310 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.292313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.292455 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:06:48.292963 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.292964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.292973 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.293029 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.294054 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.294733 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.295949 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.296276 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.296357 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.298726 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.298727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.298728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.298736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.300391 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.300813 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.300815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.300815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.300884 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.301126 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.301127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.301127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.301297 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.301394 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.301669 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.301774 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:06:48.302561 27193 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:48.302680 27193 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:48.302868 27193 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:48.303493 26932 ts_manager.cc:194] Registered new tserver with Master: 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
I20260808 07:06:48.304457 26932 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:41819
WARNING: no policy specified for oryx/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
I20260808 07:06:48.323676 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:48.407958 27199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:48.408167 27199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:48.408190 27199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:48.409372 27199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:48.409413 27199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:48.409440 27199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:48.410944 27199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:48.411221 27199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:48.411471 27199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:48.416600 27209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.416662 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.416601 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.416931 27199 server_base.cc:1034] running on GCE node
I20260808 07:06:48.417152 27199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:48.417433 27199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:48.418607 27199 hybrid_clock.cc:648] HybridClock initialized: now 1786172808418591 us; error 29 us; skew 500 ppm
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.421131 27199 init.cc:377] Logged in from keytab as oryx/127.18.63.66@KRBTEST.COM (short username oryx)
I20260808 07:06:48.421638 27199 webserver.cc:460] Webserver started at http://127.18.63.66:34937/ using document root <none> and password file <none>
I20260808 07:06:48.421902 27199 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.421972 27199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.422137 27199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:48.423195 27199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "aff17fccf9fb401b8390ddf41368827c"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c298b2b8bc09f4d3ca6c6491382c5259"
server_key_iv: "13caf52077bae33c76739b08008bc6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.423563 27199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "aff17fccf9fb401b8390ddf41368827c"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c298b2b8bc09f4d3ca6c6491382c5259"
server_key_iv: "13caf52077bae33c76739b08008bc6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.424880 27199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:48.425688 27216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.425858 27199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:48.425968 27199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "aff17fccf9fb401b8390ddf41368827c"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c298b2b8bc09f4d3ca6c6491382c5259"
server_key_iv: "13caf52077bae33c76739b08008bc6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.426043 27199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.441303 27199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.443166 27199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:48.443338 27199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.443572 27199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:48.443888 27199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:48.443944 27199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.443985 27199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:48.444015 27199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.450487 27199 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:40055
I20260808 07:06:48.450541 27369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:40055 every 8 connection(s)
I20260808 07:06:48.450845 27199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:48.459266 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27199
I20260808 07:06:48.459372 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:48.459587 18685 external_mini_cluster.cc:1428] Setting key e8b298929623def9e0464ebb12067873
I20260808 07:06:48.459678 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.451480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:52623 (local address 127.18.63.126:42871)
0808 07:06:48.451566 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.451568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.451795 (+ 227us) server_negotiation.cc:407] Connection header received
0808 07:06:48.452293 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.452294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.452305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.452358 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.453237 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.453823 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.454781 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.454966 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.455015 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.456984 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.456985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.456985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.456991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.458408 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.458762 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.458763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.458763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.458814 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.459018 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.459019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.459019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.459114 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.459163 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.459519 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.459584 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:06:48.459900 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.451443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:52623)
0808 07:06:48.451625 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.451662 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.452206 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.452409 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.452412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.452966 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.453160 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.453165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.453906 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.453907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.454706 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.454708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.455036 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.455036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.455152 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.455582 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.455590 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.456901 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.458484 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.458490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.458492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.458700 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.458871 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.458872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.458872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.458965 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.459178 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.459186 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.459361 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.459484 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.459655 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260808 07:06:48.460377 27371 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:48.460493 27371 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:48.460717 27371 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:48.461112 26931 ts_manager.cc:194] Registered new tserver with Master: aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055)
I20260808 07:06:48.461573 26931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:52623
WARNING: no policy specified for oryx/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
I20260808 07:06:48.486685 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:48.570744 27378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:48.570971 27378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:48.571012 27378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:48.572184 27378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:48.572253 27378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:48.572299 27378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:48.573813 27378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:48.574079 27378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:48.574321 27378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:48.579541 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.579530 27387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.579530 27390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.579826 27378 server_base.cc:1034] running on GCE node
I20260808 07:06:48.580034 27378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:48.580235 27378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:48.581404 27378 hybrid_clock.cc:648] HybridClock initialized: now 1786172808581380 us; error 34 us; skew 500 ppm
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.583889 27378 init.cc:377] Logged in from keytab as oryx/127.18.63.67@KRBTEST.COM (short username oryx)
I20260808 07:06:48.584378 27378 webserver.cc:460] Webserver started at http://127.18.63.67:34803/ using document root <none> and password file <none>
I20260808 07:06:48.584605 27378 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.584677 27378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.584832 27378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:48.585966 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "e6d1ce42008c48e3bb5dddf51b495626"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "655c4464297d9dc9ef87dd1ed2e0be9f"
server_key_iv: "4ae402d5fd0230360ebd738a50860b8e"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.586342 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "e6d1ce42008c48e3bb5dddf51b495626"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "655c4464297d9dc9ef87dd1ed2e0be9f"
server_key_iv: "4ae402d5fd0230360ebd738a50860b8e"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.587685 27378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:48.588467 27397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.588658 27378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:48.588763 27378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "e6d1ce42008c48e3bb5dddf51b495626"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "655c4464297d9dc9ef87dd1ed2e0be9f"
server_key_iv: "4ae402d5fd0230360ebd738a50860b8e"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.588852 27378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.601663 27378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.603612 27378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:48.603778 27378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.604007 27378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:48.604337 27378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:48.604413 27378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.604455 27378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:48.604483 27378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.610658 27378 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:37761
I20260808 07:06:48.610718 27548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:37761 every 8 connection(s)
I20260808 07:06:48.611013 27378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:48.612057 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27378
I20260808 07:06:48.612159 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:48.612342 18685 external_mini_cluster.cc:1428] Setting key 4f766e4e0357b7e3c5adf734f8ca94b5
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:48.620453 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.611771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:38461 (local address 127.18.63.126:42871)
0808 07:06:48.611855 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.611857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.612046 (+ 189us) server_negotiation.cc:407] Connection header received
0808 07:06:48.612767 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.612768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.612777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.612867 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.614171 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.614679 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.615641 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.615826 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.615878 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.617908 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.617909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.617909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.617916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.619359 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.619695 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.619696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.619697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.619751 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.619954 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.619955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.619955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.620055 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.620106 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.620330 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.620390 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:06:48.620654 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.611729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:38461)
0808 07:06:48.611903 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.611944 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.612662 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.613181 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.613186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.613870 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.614062 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.614068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.614763 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.614764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.615557 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.615559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.615896 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.615897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.615998 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.616440 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.616448 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.617821 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.619427 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.619433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.619434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.619633 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.619809 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.619809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.619810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.619902 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.620108 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.620113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.620224 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.620304 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.620416 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260808 07:06:48.621248 27549 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:48.621374 27549 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:48.621554 27549 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:48.621956 26931 ts_manager.cc:194] Registered new tserver with Master: e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:48.622291 26931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:38461
I20260808 07:06:48.624042 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.125.keytab.
Entry for principal oryx/127.18.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.125.keytab.
I20260808 07:06:48.650286 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45135
--webserver_interface=127.18.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.125.keytab
--principal=oryx/127.18.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42871,127.18.63.125:45135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:48.734259 27557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:48.734508 27557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:48.734547 27557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:48.735770 27557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:48.735837 27557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:48.735860 27557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:48.735881 27557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:48.735909 27557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:48.737407 27557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42871,127.18.63.125:45135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.125.keytab
--principal=oryx/127.18.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
W20260808 07:06:48.737551 27557 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.63.126:42871,127.18.63.125:45135'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 07:06:48.737725 27557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:48.737950 27557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:48.743077 27566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.743093 27563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.743093 27562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.743276 27557 server_base.cc:1034] running on GCE node
I20260808 07:06:48.743510 27557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:48.743840 27557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:48.745021 27557 hybrid_clock.cc:648] HybridClock initialized: now 1786172808745008 us; error 44 us; skew 500 ppm
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.747738 27557 init.cc:377] Logged in from keytab as oryx/127.18.63.125@KRBTEST.COM (short username oryx)
I20260808 07:06:48.748202 27557 webserver.cc:460] Webserver started at http://127.18.63.125:43263/ using document root <none> and password file <none>
I20260808 07:06:48.748466 27557 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.748530 27557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.748694 27557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:48.749832 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/instance:
uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c9bb2d12a76027dafe03fefdb723376d"
server_key_iv: "85d138e4c889234aec3bdcf5393afded"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.750202 27557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal/instance:
uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c9bb2d12a76027dafe03fefdb723376d"
server_key_iv: "85d138e4c889234aec3bdcf5393afded"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.751536 27557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:48.752272 27576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.752444 27557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:48.752553 27557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c9bb2d12a76027dafe03fefdb723376d"
server_key_iv: "85d138e4c889234aec3bdcf5393afded"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.752640 27557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.763605 27557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.765916 27557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:48.766083 27557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.770326 27557 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.125:45135
I20260808 07:06:48.770378 27641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.125:45135 every 8 connection(s)
I20260808 07:06:48.770771 27557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
I20260808 07:06:48.771801 27642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:48.773317 27642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:48.775679 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27557
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.125@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
I20260808 07:06:48.784152 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.774160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.18.63.126:42871)
0808 07:06:48.774234 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.774236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.774470 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:06:48.774941 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.774942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.774952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.775009 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.775683 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.776394 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.777673 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.777864 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.777919 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.780539 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.780541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.780541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.780549 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.782159 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.782678 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.782680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.782680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.782762 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.783105 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.783106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.783106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.783332 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.783402 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.783959 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.784046 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 07:06:48.784260 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.774137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36628)
0808 07:06:48.774337 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.774380 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.774859 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.775061 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.775064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.775317 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.775586 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.775593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.776491 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.776492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.777584 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.777587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.777942 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.777943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.778077 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.778575 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.778585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.780453 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.782245 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.782253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.782254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.782561 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.782855 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.782856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.782856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.783000 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.783434 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.783446 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.783563 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.783694 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.783869 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260808 07:06:48.784987 27642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:48.787043 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.776199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.18.63.125:45135)
0808 07:06:48.776499 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.776504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.776522 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:06:48.776544 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.776546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.776575 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.776663 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.777255 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.778235 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.779321 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.779814 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.779892 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.782885 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.782892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.782893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.782899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.785046 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.785812 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.785815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.785815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.785882 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.786111 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.786112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.786112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.786322 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.786873 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.786913 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.786975 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":269,"thread_start_us":78,"threads_started":1}
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.125@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
I20260808 07:06:48.792245 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.785278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41114)
0808 07:06:48.785331 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.785359 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.785400 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.785690 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.785690 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.785863 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.785923 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.785925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.786785 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.786786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.787413 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.787415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.787864 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.787865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.787930 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.788238 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.788242 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.789301 (+ 1059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.791227 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.791228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.791229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.791390 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.791634 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.791635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.791635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.791696 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.791938 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.791939 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.791977 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.792052 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.792187 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:06:48.792371 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.785319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.18.63.125:45135)
0808 07:06:48.785519 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.785521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.785534 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:06:48.785580 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.785580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.785590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.785677 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.786007 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.786699 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.787504 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.787780 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.787863 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.789419 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.789420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.789421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.789427 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.791132 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.791474 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.791475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.791476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.791552 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.791760 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.791760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.791761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.791872 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.791946 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.792100 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.792257 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":183,"thread_start_us":130,"threads_started":1}
I20260808 07:06:48.793681 27642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Bootstrap starting.
I20260808 07:06:48.794329 27642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:48.794628 27642 log.cc:824] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:48.795488 27642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: No bootstrap required, opened a new log
I20260808 07:06:48.796912 27642 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } }
I20260808 07:06:48.797056 27642 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:48.797125 27642 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8ed3e8eccbd4deaacd3b51f30660bdd, State: Initialized, Role: FOLLOWER
I20260808 07:06:48.797276 27642 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } }
I20260808 07:06:48.797637 27642 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:48.797608 27649 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } } }
I20260808 07:06:48.797713 27649 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:48.799710 27662 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:48.799768 27662 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:48.801054 27557 master_runner.cc:428] Detected that this master d8ed3e8eccbd4deaacd3b51f30660bdd is joining an existing cluster
I20260808 07:06:48.801121 27557 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.63.125:45135
I20260808 07:06:48.808919 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.802265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.18.63.126:42871)
0808 07:06:48.802316 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.802319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.802619 (+ 300us) server_negotiation.cc:407] Connection header received
0808 07:06:48.802689 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.802690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.802701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.802754 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.803093 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.803588 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.804386 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.804596 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.804690 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.805887 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.805888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.805888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.805896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.807716 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.808092 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.808093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.808094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.808164 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.808386 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.808387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.808387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.808508 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.808590 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.808746 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.808823 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:48.808933 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.802325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36638)
0808 07:06:48.802533 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.802542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.802625 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.802756 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.802756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.802937 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.803004 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.803006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.803692 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.803693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.804296 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.804298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.804714 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.804715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.804784 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.805259 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.805265 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.805805 (+ 540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.807809 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.807810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.807810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.808013 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.808237 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.808238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.808239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.808302 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.808582 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.808583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.808628 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.808720 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.808824 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260808 07:06:48.812530 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.802383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.18.63.125:45135)
0808 07:06:48.802429 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.802431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.805525 (+ 3094us) server_negotiation.cc:407] Connection header received
0808 07:06:48.805577 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.805578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.805588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.805659 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.806037 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.806771 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.808354 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.808679 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.808750 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.809868 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.809869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.809869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.809876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.811482 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.811845 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.811846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.811846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.811890 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.812057 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.812057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.812058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.812169 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.812209 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.812374 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.812443 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:06:48.812821 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.802403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41124)
0808 07:06:48.805440 (+ 3037us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.805452 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.805518 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.805662 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.805663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.805896 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.805956 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.805958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.806879 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.806880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.808263 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.808265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.808796 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.808797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.808859 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.809364 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.809369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.809782 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.811570 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.811571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.811571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.811773 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.811952 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.811952 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.811952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.811998 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.812249 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.812250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.812281 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.812645 (+ 364us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.812725 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3019,"thread_start_us":83,"threads_started":1}
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.126@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
I20260808 07:06:48.819502 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.812087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.18.63.125:45135)
0808 07:06:48.812137 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.812139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.812326 (+ 187us) server_negotiation.cc:407] Connection header received
0808 07:06:48.812419 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.812420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.812430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.812488 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.813024 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.813724 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.814540 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.814771 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.814870 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.816826 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.816828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.816828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.816835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.818274 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.818686 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.818687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.818687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.818743 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.818952 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.818953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.818953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.819064 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.819140 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.819330 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.819409 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:06:48.819530 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.812056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41130)
0808 07:06:48.812191 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.812247 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.812351 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.812493 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.812496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.812852 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.812925 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.812928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.813824 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.813824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.814449 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.814451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.814871 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.814872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.814958 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.815433 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.815438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.816728 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.818379 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.818387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.818389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.818594 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.818818 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.818818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.818819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.818880 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.819135 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.819144 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.819197 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.819291 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.819413 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260808 07:06:48.820114 26931 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.63.125:45135: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
member_type: NON_VOTER
last_known_addr {
host: "127.18.63.125"
port: 45135
}
attrs {
promote: true
}
}
dest_uuid: "928f91425d204020b989476350668cf6"
cas_config_opid_index: -1
I20260808 07:06:48.820464 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } }
I20260808 07:06:48.820982 27676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } }
I20260808 07:06:48.821324 27610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:48.822237 27676 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.63.125:45135
I20260808 07:06:48.822327 27610 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 07:06:48.822333 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } }
I20260808 07:06:48.822392 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:48.822587 27557 master.cc:537] Master@127.18.63.125:45135 shutting down...
I20260808 07:06:48.822642 27677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:06:48.823994 26916 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 07:06:48.824415 26916 consensus_peers.cc:487] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 -> Peer d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125:45135): Couldn't send request to peer d8ed3e8eccbd4deaacd3b51f30660bdd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:06:48.825439 27557 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:48.825531 27557 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:48.825580 27557 tablet_replica.cc:330] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: stopping tablet replica
I20260808 07:06:48.838675 27557 master.cc:552] Master@127.18.63.125:45135 shutdown complete.
I20260808 07:06:48.838728 27557 master_runner.cc:305] Clearing existing system tablet
I20260808 07:06:48.838864 27557 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.838914 27557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.839637 27557 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260808 07:06:48.840162 27681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.840315 27557 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.840378 27557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.840404 27557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c9bb2d12a76027dafe03fefdb723376d"
server_key_iv: "85d138e4c889234aec3bdcf5393afded"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.841226 27557 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:48.842785 27557 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:06:48.842860 27557 log.cc:1196] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 07:06:48.843065 27557 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Deleting consensus metadata
I20260808 07:06:48.843189 27557 master_runner.cc:315] Copying system tablet from 127.18.63.126:42871
I20260808 07:06:48.843812 27557 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: tablet copy: Beginning tablet copy session from remote peer at address 127.18.63.126:42871
I20260808 07:06:48.850274 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.844158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.18.63.126:42871)
0808 07:06:48.844205 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.844207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.844272 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:06:48.844335 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.844335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.844344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.844392 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.844749 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.845240 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.846335 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.846529 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.846583 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.847668 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.847670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.847670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.847676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.849148 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.849610 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.849611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.849612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.849662 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.849855 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.849855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.849855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.849945 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.849992 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.850154 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.850211 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:48.850361 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.844058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36652)
0808 07:06:48.844190 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:48.844201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:48.844270 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:48.844397 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:48.844398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:48.844597 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:48.844676 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.844678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.845319 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.845320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.846254 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:48.846257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.846602 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.846603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:48.846682 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.847176 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:48.847182 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:48.847577 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:48.849232 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.849233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.849233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.849535 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.849726 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:48.849727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:48.849727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:48.849796 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:48.850005 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:48.850006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:48.850056 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 07:06:48.850136 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:06:48.850242 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":89,"threads_started":1}
I20260808 07:06:48.850966 26962 tablet_copy_service.cc:140] P 928f91425d204020b989476350668cf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d8ed3e8eccbd4deaacd3b51f30660bdd ({username='oryx', principal='oryx/127.18.63.125@KRBTEST.COM'} at 127.0.0.1:36652)
I20260808 07:06:48.851076 26962 tablet_copy_service.cc:161] P 928f91425d204020b989476350668cf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d8ed3e8eccbd4deaacd3b51f30660bdd at {username='oryx', principal='oryx/127.18.63.125@KRBTEST.COM'} at 127.0.0.1:36652: session id = d8ed3e8eccbd4deaacd3b51f30660bdd-00000000000000000000000000000000
I20260808 07:06:48.851761 26962 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:06:48.852415 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:06:48.853739 27557 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: tablet copy: Starting download of 0 data blocks...
I20260808 07:06:48.853888 27557 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: tablet copy: Starting download of 1 WAL segments...
I20260808 07:06:48.854859 27557 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:06:48.855774 26962 tablet_copy_service.cc:342] P 928f91425d204020b989476350668cf6: Request end of tablet copy session d8ed3e8eccbd4deaacd3b51f30660bdd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.63.125@KRBTEST.COM'} at 127.0.0.1:36652
I20260808 07:06:48.855835 26962 tablet_copy_service.cc:433] P 928f91425d204020b989476350668cf6: ending tablet copy session d8ed3e8eccbd4deaacd3b51f30660bdd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d8ed3e8eccbd4deaacd3b51f30660bdd
I20260808 07:06:48.858353 27557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:48.859680 27691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:48.859773 27693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.859695 27557 server_base.cc:1034] running on GCE node
W20260808 07:06:48.859683 27690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:48.860047 27557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:48.860224 27557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:48.861377 27557 hybrid_clock.cc:648] HybridClock initialized: now 1786172808861367 us; error 27 us; skew 500 ppm
Aug 08 07:06:48 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172808, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:48.862711 27557 init.cc:377] Logged in from keytab as oryx/127.18.63.125@KRBTEST.COM (short username oryx)
I20260808 07:06:48.863040 27557 webserver.cc:460] Webserver started at http://127.18.63.125:44611/ using document root <none> and password file <none>
I20260808 07:06:48.863176 27557 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:48.863219 27557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:48.863847 27557 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:48.864406 27701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:48.864562 27557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:48.864643 27557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c9bb2d12a76027dafe03fefdb723376d"
server_key_iv: "85d138e4c889234aec3bdcf5393afded"
server_key_version: "encryptionkey@0"
I20260808 07:06:48.864706 27557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:48.872797 27557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:48.874650 27557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:48.877393 27557 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.125:45135
I20260808 07:06:48.877713 27769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.125:45135 every 8 connection(s)
I20260808 07:06:48.877838 27557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
I20260808 07:06:48.878613 27771 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:06:48.879074 27771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Bootstrap starting.
I20260808 07:06:48.881775 27771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:06:48.882056 27771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: Bootstrap complete.
I20260808 07:06:48.882283 27771 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } }
I20260808 07:06:48.882355 27771 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d8ed3e8eccbd4deaacd3b51f30660bdd, State: Initialized, Role: LEARNER
I20260808 07:06:48.882385 27771 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } }
I20260808 07:06:48.882632 27771 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:48.882596 27774 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } }
I20260808 07:06:48.882802 27774 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: LEARNER
I20260808 07:06:48.884182 27789 catalog_manager.cc:1259] Loaded cluster ID: 47124f5937ad452c84cf1f8f5c4f010f
I20260808 07:06:48.884253 27789 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: loading cluster ID for follower catalog manager: success
I20260808 07:06:48.885234 27789 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: acquiring CA information for follower catalog manager: success
I20260808 07:06:48.885564 27789 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:48.895793 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:48.888539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.18.63.125:45135)
0808 07:06:48.888649 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:48.888651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:48.888666 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:06:48.888692 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:48.888693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:48.888704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:48.888801 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:48.889331 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.890402 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.891192 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:48.891499 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:48.891582 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:48.892898 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:48.892900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:48.892900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:48.892908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:48.894617 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.894967 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.894968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.894969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.895036 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:48.895232 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:48.895233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:48.895233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:48.895368 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:48.895478 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:48.895588 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 07:06:48.895682 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
WARNING: no policy specified for oryx/127.18.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.124.keytab.
Entry for principal oryx/127.18.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.124.keytab.
I20260808 07:06:48.924388 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:39067
--webserver_interface=127.18.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.124.keytab
--principal=oryx/127.18.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:49.010667 27793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:49.010880 27793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:49.010902 27793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:49.012079 27793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:49.012125 27793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:49.012140 27793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:49.012192 27793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:49.012217 27793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:49.014034 27793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.124.keytab
--principal=oryx/127.18.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:39067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:49.014307 27793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:49.014549 27793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:49.019630 27801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.019779 27803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.019886 27800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.019929 27793 server_base.cc:1034] running on GCE node
I20260808 07:06:49.020186 27793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:49.020445 27793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:49.021579 27793 hybrid_clock.cc:648] HybridClock initialized: now 1786172809021570 us; error 29 us; skew 500 ppm
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:49.024154 27793 init.cc:377] Logged in from keytab as oryx/127.18.63.124@KRBTEST.COM (short username oryx)
I20260808 07:06:49.024657 27793 webserver.cc:460] Webserver started at http://127.18.63.124:38595/ using document root <none> and password file <none>
I20260808 07:06:49.024902 27793 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.024971 27793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.025127 27793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:49.026280 27793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/instance:
uuid: "b1a4105367784ede815cd51cba247166"
format_stamp: "Formatted at 2026-08-08 07:06:49 on dist-test-slave-bw88"
server_key: "ab6d571f03c89e3bea87db3d91761e89"
server_key_iv: "ada3c5d10ace4217d6a33bbd7c86ed5b"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.026669 27793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal/instance:
uuid: "b1a4105367784ede815cd51cba247166"
format_stamp: "Formatted at 2026-08-08 07:06:49 on dist-test-slave-bw88"
server_key: "ab6d571f03c89e3bea87db3d91761e89"
server_key_iv: "ada3c5d10ace4217d6a33bbd7c86ed5b"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.028008 27793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:49.028756 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.028941 27793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:49.029048 27793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "b1a4105367784ede815cd51cba247166"
format_stamp: "Formatted at 2026-08-08 07:06:49 on dist-test-slave-bw88"
server_key: "ab6d571f03c89e3bea87db3d91761e89"
server_key_iv: "ada3c5d10ace4217d6a33bbd7c86ed5b"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.029125 27793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.073128 27793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:49.075670 27793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:49.075852 27793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:49.080179 27793 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.124:39067
I20260808 07:06:49.080226 27877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.124:39067 every 8 connection(s)
I20260808 07:06:49.080646 27793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
I20260808 07:06:49.081848 27878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:49.083765 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:06:49.090348 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27793
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.124@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:49.095456 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.084654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.18.63.126:42871)
0808 07:06:49.084767 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.084770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.084951 (+ 181us) server_negotiation.cc:407] Connection header received
0808 07:06:49.086033 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.086034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.086048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.086128 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.086806 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.087323 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.088361 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.088694 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.088777 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.091728 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.091729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.091730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.091737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.093628 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.094174 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.094176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.094176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.094247 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.094577 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.094578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.094578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.094791 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.094860 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.095266 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.095360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 07:06:49.095710 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.084534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36664)
0808 07:06:49.084786 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.084839 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.085872 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.086210 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.086214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.086499 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.086704 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.086708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.087429 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.087431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.088254 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.088256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.088821 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.088823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.089012 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.089680 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.089691 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.091640 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.093709 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.093717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.093720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.094063 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.094331 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.094332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.094333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.094482 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.094875 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.094883 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.095094 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.095230 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.095407 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":127,"threads_started":1}
I20260808 07:06:49.096527 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.124@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
I20260808 07:06:49.101384 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.091055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.18.63.124:39067)
0808 07:06:49.091390 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.091395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.091413 (+ 18us) server_negotiation.cc:407] Connection header received
0808 07:06:49.091435 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.091437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.091457 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.091543 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.092610 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.093538 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.094667 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.095175 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.096023 (+ 848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.098207 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.098214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.098214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.098221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.100052 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.100419 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.100422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.100423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.100488 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.100724 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.100724 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.100725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.100863 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.100964 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.101097 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.101218 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":102,"threads_started":1}
I20260808 07:06:49.104099 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.096848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41148)
0808 07:06:49.096910 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.096923 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.096972 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.097148 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.097149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.097261 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.097310 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.097311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.098294 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.098295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.099188 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.099191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.099562 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.099563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.099640 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.099955 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.099961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.101566 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.103120 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.103120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.103121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.103292 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.103510 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.103515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.103515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.103600 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.103851 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.103852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.103892 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.103945 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.104027 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:06:49.104105 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.096916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.18.63.125:45135)
0808 07:06:49.096962 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.096964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.096973 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:49.097054 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.097055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.097073 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.097145 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.097427 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.098193 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.099280 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.099480 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.099540 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.101693 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.101695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.101695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.101704 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.103045 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.103352 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.103353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.103353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.103403 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.103682 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.103683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.103683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.103791 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.103845 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.103963 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.104041 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:49.104710 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.124@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
I20260808 07:06:49.110864 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.104988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39430)
0808 07:06:49.105041 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.105059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.105111 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.105228 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.105229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.105385 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.105423 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.105428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.106007 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.106008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.106481 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.106482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.106818 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.106819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.106855 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.107264 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.107269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.108369 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.110068 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.110069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.110069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.110229 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.110385 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.110385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.110385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.110426 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.110637 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.110638 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.110674 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.110731 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.110802 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:06:49.110993 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.105030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.18.63.124:39067)
0808 07:06:49.105064 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.105066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.105096 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:06:49.105167 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.105167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.105175 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.105214 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.105482 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.105944 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.106550 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.106747 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.106811 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.108441 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.108443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.108443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.108449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.109979 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.110294 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.110295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.110295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.110344 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.110458 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.110459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.110459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.110546 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.110593 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.110768 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.110848 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":17}
I20260808 07:06:49.112650 27878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Bootstrap starting.
I20260808 07:06:49.113660 27878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:49.113981 27878 log.cc:824] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:49.115042 27878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: No bootstrap required, opened a new log
I20260808 07:06:49.116539 27878 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } }
I20260808 07:06:49.116699 27878 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:49.116760 27878 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a4105367784ede815cd51cba247166, State: Initialized, Role: FOLLOWER
I20260808 07:06:49.116907 27878 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } }
I20260808 07:06:49.117225 27878 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:49.117235 27884 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } } }
I20260808 07:06:49.117519 27884 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:06:49.119416 27896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:06:49.119488 27896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:06:49.121698 27793 master_runner.cc:428] Detected that this master b1a4105367784ede815cd51cba247166 is joining an existing cluster
I20260808 07:06:49.121740 27793 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.63.124:39067
I20260808 07:06:49.131470 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.18.63.126:42871)
0808 07:06:49.123240 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.123242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.123525 (+ 283us) server_negotiation.cc:407] Connection header received
0808 07:06:49.123636 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.123637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.123649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.123680 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.124257 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.125043 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.127365 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.127684 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.127727 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.128847 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.128848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.128848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.128855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.130344 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.130747 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.130748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.130748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.130799 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.131021 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.131022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.131022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.131132 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.131183 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.131345 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.131405 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:49.131963 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36668)
0808 07:06:49.123458 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.123467 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.123548 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.123744 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.123745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.124068 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.124168 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.124169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.126041 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.126909 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.126912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.127810 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.127810 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.127881 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.128231 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.128237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.128716 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.130428 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.130429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.130430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.130666 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.130860 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.130861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.130861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.130957 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.131194 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.131195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.131243 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.131325 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.131429 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":7,"thread_start_us":79,"threads_started":1}
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172807, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.126@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
I20260808 07:06:49.152945 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27022
I20260808 07:06:49.166667 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.18.63.125:45135)
0808 07:06:49.123281 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.123283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.123485 (+ 202us) server_negotiation.cc:407] Connection header received
0808 07:06:49.123598 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.123599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.123612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.123646 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.125165 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.125927 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126935 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.127183 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.127256 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.161739 (+ 34483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.161741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.161741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.161750 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.165178 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165794 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.165795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.165795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.165861 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.166066 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.166067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.166067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.166197 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.166251 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.166496 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.166574 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:49.166692 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39440)
0808 07:06:49.124280 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.124288 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.124365 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.124490 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.124491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.124660 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.124707 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.124709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.125540 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.125541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.126246 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.126248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.126725 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.126799 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.127734 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.127739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.163211 (+ 35472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.165375 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.165377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.165704 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.165918 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.165919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.165986 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.166273 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.166273 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.166328 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.166434 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.166564 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260808 07:06:49.167053 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.18.63.124:39067)
0808 07:06:49.123556 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.123558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.124355 (+ 797us) server_negotiation.cc:407] Connection header received
0808 07:06:49.124410 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.124411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.124420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.124474 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.124790 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.125462 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126339 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.126614 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126650 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.163311 (+ 36661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.163312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.163312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.163319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.165276 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165785 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.165786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.165787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.165852 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.166051 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.166052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.166052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.166186 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.166250 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.166889 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.166975 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:06:49.167671 27906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.123186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41158)
0808 07:06:49.123413 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.123425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.123515 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.123724 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.123725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.124065 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.124166 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.124169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.126016 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.126017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.126840 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.126843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.127334 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.127335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.127398 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.161088 (+ 33690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.161096 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.161610 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.165284 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.165286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.165704 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.165930 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.165931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.165931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.165998 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.166275 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.166275 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.166330 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.166434 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.167541 (+ 1107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":997,"thread_start_us":95,"threads_started":1}
I20260808 07:06:49.169512 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.134039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.18.63.124:39067)
0808 07:06:49.134218 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.134219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.134234 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:06:49.134264 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.134264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.134272 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.134333 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.134638 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.135118 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.136088 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.136280 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.136325 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.138374 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.138376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.138376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.138383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.168204 (+ 29821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.168645 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.168646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.168646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.168702 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.168935 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.168936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.168936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.169049 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.169131 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.169300 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.169410 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260808 07:06:49.169512 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.133988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39452)
0808 07:06:49.134042 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.134052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.134088 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.134354 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.134355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.134521 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.134565 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.134567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.135190 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.135191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.136002 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.136004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.136344 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.136345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.136414 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.136783 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.136789 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.138257 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.168324 (+ 30067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.168326 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.168327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.168561 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.168783 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.168784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.168785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.168855 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.169128 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.169130 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.169189 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.169274 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.169412 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:06:49.170073 26931 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.63.124:39067: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b1a4105367784ede815cd51cba247166"
member_type: NON_VOTER
last_known_addr {
host: "127.18.63.124"
port: 39067
}
attrs {
promote: true
}
}
dest_uuid: "928f91425d204020b989476350668cf6"
cas_config_opid_index: 5
I20260808 07:06:49.170197 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:46189
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:42871
--webserver_port=38275
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:49.170385 26931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } }
I20260808 07:06:49.170732 27676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b1a4105367784ede815cd51cba247166 (127.18.63.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.171633 27676 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.63.124:39067
I20260808 07:06:49.171708 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.171788 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.172374 27793 master.cc:537] Master@127.18.63.124:39067 shutting down...
I20260808 07:06:49.172591 27847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:49.173790 27847 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:06:49.174068 27677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:06:49.174899 26917 consensus_peers.cc:487] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 -> Peer b1a4105367784ede815cd51cba247166 (127.18.63.124:39067): Couldn't send request to peer b1a4105367784ede815cd51cba247166. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 07:06:49.175619 27793 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:49.175711 27793 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:49.175757 27793 tablet_replica.cc:330] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: stopping tablet replica
I20260808 07:06:49.177779 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.171977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41160)
0808 07:06:49.174047 (+ 2070us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.174060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.174105 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.174347 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.174348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.174350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.174398 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.174400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.175318 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.175319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.176533 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.176537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.177480 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.177481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.177556 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.177582 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.177649 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.177696 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2052}
I20260808 07:06:49.177999 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.172085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.18.63.125:45135)
0808 07:06:49.172136 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.172138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.174159 (+ 2021us) server_negotiation.cc:407] Connection header received
0808 07:06:49.174203 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.174204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.174231 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.174288 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.174479 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.175218 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.176645 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.177289 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.177371 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.177440 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.177873 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.177917 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:49.178242 27737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 LEARNER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 07:06:49.178429 27677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 07:06:49.179039 27737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b1a4105367784ede815cd51cba247166 (127.18.63.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.180032 27774 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928f91425d204020b989476350668cf6. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.180121 27774 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: LEARNER
I20260808 07:06:49.180167 27774 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: NON_VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: true } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.180218 27774 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: LEARNER
I20260808 07:06:49.180266 27677 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: attempting to promote NON_VOTER d8ed3e8eccbd4deaacd3b51f30660bdd to VOTER
I20260808 07:06:49.180461 27677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } }
I20260808 07:06:49.181013 27737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 LEARNER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 07:06:49.181267 27914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 07:06:49.181327 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.180796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39458)
0808 07:06:49.181260 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.181307 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447}
I20260808 07:06:49.181733 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.181630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39462)
0808 07:06:49.181678 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.181707 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:06:49.181938 26917 consensus_peers.cc:487] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 -> Peer b1a4105367784ede815cd51cba247166 (127.18.63.124:39067): Couldn't send request to peer b1a4105367784ede815cd51cba247166. Status: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:06:49.182190 27677 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.183040 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.183130 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.183205 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.183267 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.183493 27737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.184345 27913 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.184440 27913 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:49.189810 27793 master.cc:552] Master@127.18.63.124:39067 shutdown complete.
I20260808 07:06:49.189874 27793 master_runner.cc:305] Clearing existing system tablet
I20260808 07:06:49.190009 27793 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.190073 27793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.190855 27793 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:49.191365 27917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.191505 27793 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.191565 27793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.191596 27793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "b1a4105367784ede815cd51cba247166"
format_stamp: "Formatted at 2026-08-08 07:06:49 on dist-test-slave-bw88"
server_key: "ab6d571f03c89e3bea87db3d91761e89"
server_key_iv: "ada3c5d10ace4217d6a33bbd7c86ed5b"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.192329 27793 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:49.193810 27793 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 07:06:49.193897 27793 log.cc:1196] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 07:06:49.194123 27793 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Deleting consensus metadata
I20260808 07:06:49.194267 27793 master_runner.cc:315] Copying system tablet from 127.18.63.126:42871
I20260808 07:06:49.194911 27793 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: tablet copy: Beginning tablet copy session from remote peer at address 127.18.63.126:42871
I20260808 07:06:49.201318 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.195317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:36682)
0808 07:06:49.195430 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.195442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.195537 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.195669 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.195670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.195863 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.195916 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.195918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.196689 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.196690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.197239 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.197241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.197649 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.197650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.197720 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.198218 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.198224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.198536 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.200222 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.200223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.200223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.200409 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.200633 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.200634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.200634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.200699 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.200960 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.200961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.201008 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.201094 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.201217 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260808 07:06:49.201311 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.195255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.18.63.126:42871)
0808 07:06:49.195300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.195302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.195512 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:06:49.195602 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.195603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.195615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.195666 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.196000 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.196589 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.197357 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.197570 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.197649 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.198654 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.198655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.198656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.198664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.200116 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.200484 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.200485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.200485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.200550 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.200769 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.200770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.200770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.200884 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.200957 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.201127 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.201216 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:49.201987 26962 tablet_copy_service.cc:140] P 928f91425d204020b989476350668cf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b1a4105367784ede815cd51cba247166 ({username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:36682)
I20260808 07:06:49.202070 26962 tablet_copy_service.cc:161] P 928f91425d204020b989476350668cf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b1a4105367784ede815cd51cba247166 at {username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:36682: session id = b1a4105367784ede815cd51cba247166-00000000000000000000000000000000
I20260808 07:06:49.202531 26962 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 07:06:49.203006 27793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 07:06:49.203912 27793 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: tablet copy: Starting download of 0 data blocks...
I20260808 07:06:49.204087 27793 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: tablet copy: Starting download of 1 WAL segments...
I20260808 07:06:49.204785 27793 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 07:06:49.205535 26962 tablet_copy_service.cc:342] P 928f91425d204020b989476350668cf6: Request end of tablet copy session b1a4105367784ede815cd51cba247166-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:36682
I20260808 07:06:49.205610 26962 tablet_copy_service.cc:433] P 928f91425d204020b989476350668cf6: ending tablet copy session b1a4105367784ede815cd51cba247166-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b1a4105367784ede815cd51cba247166
I20260808 07:06:49.207769 27793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:49.208977 27926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.209143 27933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.209178 27928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.209777 27793 server_base.cc:1034] running on GCE node
I20260808 07:06:49.209872 27793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:49.210027 27793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:49.211151 27793 hybrid_clock.cc:648] HybridClock initialized: now 1786172809211139 us; error 30 us; skew 500 ppm
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:49.212520 27793 init.cc:377] Logged in from keytab as oryx/127.18.63.124@KRBTEST.COM (short username oryx)
I20260808 07:06:49.212813 27793 webserver.cc:460] Webserver started at http://127.18.63.124:35993/ using document root <none> and password file <none>
I20260808 07:06:49.212935 27793 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.212997 27793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.221587 27793 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.001s
I20260808 07:06:49.222204 27944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.222352 27793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:49.222404 27793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "b1a4105367784ede815cd51cba247166"
format_stamp: "Formatted at 2026-08-08 07:06:49 on dist-test-slave-bw88"
server_key: "ab6d571f03c89e3bea87db3d91761e89"
server_key_iv: "ada3c5d10ace4217d6a33bbd7c86ed5b"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.222486 27793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.244082 27793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:49.246104 27793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:49.249565 27793 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.124:39067
I20260808 07:06:49.249956 27793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
I20260808 07:06:49.250793 28008 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:06:49.251242 28008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Bootstrap starting.
I20260808 07:06:49.252384 28007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.124:39067 every 8 connection(s)
I20260808 07:06:49.254312 28008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:06:49.254552 28008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Bootstrap complete.
I20260808 07:06:49.255448 28008 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } }
I20260808 07:06:49.255564 28008 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b1a4105367784ede815cd51cba247166, State: Initialized, Role: LEARNER
I20260808 07:06:49.255609 28008 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } }
I20260808 07:06:49.255816 28008 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:49.258108 28025 catalog_manager.cc:1259] Loaded cluster ID: 47124f5937ad452c84cf1f8f5c4f010f
I20260808 07:06:49.258150 28025 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: loading cluster ID for follower catalog manager: success
I20260808 07:06:49.259307 28025 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: acquiring CA information for follower catalog manager: success
I20260808 07:06:49.259636 28025 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:06:49.259877 28011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.259974 28011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: LEARNER
W20260808 07:06:49.302690 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:49.302990 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:49.303030 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:49.304914 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:49.304980 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:49.305023 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:49.307559 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:46189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=38275
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:49.307839 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:49.308120 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:49.313968 28035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.314049 28031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.314131 28033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.314337 27911 server_base.cc:1034] running on GCE node
I20260808 07:06:49.314568 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:49.314821 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:49.315986 27911 hybrid_clock.cc:648] HybridClock initialized: now 1786172809315972 us; error 30 us; skew 500 ppm
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:49.318984 27911 init.cc:377] Logged in from keytab as oryx/127.18.63.65@KRBTEST.COM (short username oryx)
I20260808 07:06:49.319595 27911 webserver.cc:460] Webserver started at http://127.18.63.65:38275/ using document root <none> and password file <none>
I20260808 07:06:49.319882 27911 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.319962 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.321813 27911 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:49.322549 28043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.322757 27911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:49.322866 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "42095ed2821f4ddd9b3281ff509327a1"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c559d18bbdb7c15364f8739c44c07b1e"
server_key_iv: "d7991a97c7e36a256899a91fae4a3907"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.323341 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.342424 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:49.344456 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:49.344583 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:49.344790 27911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:49.345150 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:49.345213 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.345260 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:49.345288 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.352903 27911 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46189
I20260808 07:06:49.352967 28182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46189 every 8 connection(s)
I20260808 07:06:49.353375 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:49.353744 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 27911
I20260808 07:06:49.353948 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27199
I20260808 07:06:49.363389 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:40055
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:42871
--webserver_port=34937
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
I20260808 07:06:49.369796 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.356259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:60723 (local address 127.18.63.126:42871)
0808 07:06:49.356307 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.356309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.356637 (+ 328us) server_negotiation.cc:407] Connection header received
0808 07:06:49.358029 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.358031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.358049 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.358126 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.358955 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.359863 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361555 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.361932 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361983 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.365542 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.365544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.365544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.365555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.367759 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368449 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.368450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.368451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.368520 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368856 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.368857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.368858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.369019 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.369080 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.369539 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.369648 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:49.370244 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.354517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:60723)
0808 07:06:49.356566 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.356576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.357862 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.358205 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.358207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.358523 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.358828 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.358835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.359983 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.359984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.361363 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.361366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.362162 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.362163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.362248 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.362864 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.362871 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.365409 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.367864 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.367873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.367876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.368329 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.368598 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.368600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.368764 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.369118 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.369125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.369303 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.369943 (+ 640us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.370072 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2037,"mutex_wait_us":62,"thread_start_us":1370,"threads_started":1}
I20260808 07:06:49.370337 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.355981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:53385 (local address 127.18.63.125:45135)
0808 07:06:49.356029 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.356031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.356522 (+ 491us) server_negotiation.cc:407] Connection header received
0808 07:06:49.358046 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.358048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.358061 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.358093 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.359025 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.359747 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361470 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.361767 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361811 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.365993 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.365994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.365994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.366014 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.368667 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.369090 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.369091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.369092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.369155 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.369402 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.369403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.369403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.369556 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.369623 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.370149 (+ 526us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.370235 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:06:49.370079 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.354446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.18.63.65:53385)
0808 07:06:49.356449 (+ 2003us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.356459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.357925 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.358189 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.358189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.358869 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.358930 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.358932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.359868 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.359869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.361365 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.361368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.361909 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.362058 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.362966 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.362972 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.365891 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.368756 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.368757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.368996 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.369229 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.369229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.369230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.369290 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.369656 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.369657 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.369696 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.369754 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.369851 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1977,"mutex_wait_us":177,"thread_start_us":1404,"threads_started":1}
I20260808 07:06:49.370765 28183 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:49.370878 28183 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.370970 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.356132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:51045 (local address 127.18.63.124:39067)
0808 07:06:49.356381 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.356382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.356392 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:49.357713 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.357715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.357730 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.357820 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.358950 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.359984 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361463 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.361789 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361847 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.365911 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.365912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.365912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.365919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.367934 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368478 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.368480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.368480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.368552 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368850 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.368851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.368851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.369016 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.369086 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.370132 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.370240 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":70,"threads_started":1}
I20260808 07:06:49.371078 28183 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:49.371115 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.354232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.18.63.65:51045)
0808 07:06:49.356127 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.356179 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.357547 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.357933 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.357937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.358507 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.358828 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.358835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.360068 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.360069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.361365 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.361368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.361907 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.361909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.362058 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.362691 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.362701 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.365804 (+ 3103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.368033 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.368034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.368357 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.368632 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.368633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.368634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.368766 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.369116 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.369124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.369303 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.369470 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.369671 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"mutex_wait_us":71,"thread_start_us":92,"threads_started":1}
I20260808 07:06:49.371300 28185 heartbeater.cc:347] Connected to a master server at 127.18.63.125:45135
I20260808 07:06:49.371343 28185 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.371411 28185 heartbeater.cc:511] Master 127.18.63.125:45135 requested a full tablet report, sending...
I20260808 07:06:49.371480 28184 heartbeater.cc:347] Connected to a master server at 127.18.63.124:39067
I20260808 07:06:49.371539 28184 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.371572 26931 ts_manager.cc:194] Re-registered known tserver with Master: 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
I20260808 07:06:49.371613 28184 heartbeater.cc:511] Master 127.18.63.124:39067 requested a full tablet report, sending...
I20260808 07:06:49.372000 27957 ts_manager.cc:194] Registered new tserver with Master: 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
I20260808 07:06:49.372104 26931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:60723
I20260808 07:06:49.372493 27724 ts_manager.cc:194] Registered new tserver with Master: 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
W20260808 07:06:49.452394 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:49.452634 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:49.452673 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:49.453863 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:49.453931 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:49.453976 28191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:49.455600 28191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:40055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=34937
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:49.455878 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:49.456132 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:49.461472 28198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.461510 28196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.461504 28201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.461675 28191 server_base.cc:1034] running on GCE node
I20260808 07:06:49.461918 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:49.462119 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:49.463277 28191 hybrid_clock.cc:648] HybridClock initialized: now 1786172809463263 us; error 30 us; skew 500 ppm
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:49.465970 28191 init.cc:377] Logged in from keytab as oryx/127.18.63.66@KRBTEST.COM (short username oryx)
I20260808 07:06:49.466413 28191 webserver.cc:460] Webserver started at http://127.18.63.66:34937/ using document root <none> and password file <none>
I20260808 07:06:49.466671 28191 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.466760 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.468076 28191 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:49.468640 28212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.468811 28191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.468915 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "aff17fccf9fb401b8390ddf41368827c"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "c298b2b8bc09f4d3ca6c6491382c5259"
server_key_iv: "13caf52077bae33c76739b08008bc6b7"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.469260 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.487614 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:49.489519 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:49.489701 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:49.489941 28191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:49.490267 28191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:49.490325 28191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.490367 28191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:49.490408 28191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.497107 28191 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:40055
I20260808 07:06:49.497164 28360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:40055 every 8 connection(s)
I20260808 07:06:49.497540 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:49.499891 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 28191
I20260808 07:06:49.500067 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27378
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
I20260808 07:06:49.511943 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:52505 (local address 127.18.63.126:42871)
0808 07:06:49.498542 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.498545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.498620 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:06:49.499802 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.499803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.499818 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.499853 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.501006 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.501834 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503508 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.503811 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503850 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.507373 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.507375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.507375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.507384 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.509655 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.510614 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.510616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.510616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.510687 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.510987 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.510988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.510989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.511143 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.511189 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.511754 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.511841 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 07:06:49.511945 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:54651 (local address 127.18.63.125:45135)
0808 07:06:49.498346 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.498348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.498629 (+ 281us) server_negotiation.cc:407] Connection header received
0808 07:06:49.499803 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.499805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.499818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.499853 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.501007 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.501826 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503508 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.503807 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503850 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.507684 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.507686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.507686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.507694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.510499 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.510884 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.510885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.510886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.510943 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.511160 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.511161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.511161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.511292 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.511339 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.511754 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.511841 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:06:49.512902 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.18.63.66:54651)
0808 07:06:49.498480 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.498529 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.499666 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.500045 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.500049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.500810 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.500901 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.500902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.501979 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.501980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.503383 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.503386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.503947 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.504116 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.505114 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.505118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.507608 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.510588 (+ 2980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.510589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.510589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.510802 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.511023 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.511023 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.511024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.511074 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.511395 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.511403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.511584 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.512456 (+ 872us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.512645 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":44,"thread_start_us":108,"threads_started":1}
I20260808 07:06:49.513615 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:52505)
0808 07:06:49.498480 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.498529 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.499666 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.500045 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.500049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.500822 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.500890 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.500893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.501979 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.501980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.503383 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.503385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.503946 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.504116 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.504999 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.505007 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.507270 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.509887 (+ 2617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.509895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.509897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.510195 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.510768 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.510769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.510770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.510905 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.511587 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.511588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.511632 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.513389 (+ 1757us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.513502 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":83,"thread_start_us":109,"threads_started":1}
I20260808 07:06:49.513815 28367 heartbeater.cc:347] Connected to a master server at 127.18.63.125:45135
I20260808 07:06:49.513945 28367 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.514132 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:40221 (local address 127.18.63.124:39067)
0808 07:06:49.498790 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.498793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.498807 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:06:49.499814 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.499816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.499832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.499868 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.501024 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.502097 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.503563 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.503980 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.504033 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.510309 (+ 6276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.510311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.510312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.510320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.512781 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.513226 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.513228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.513228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.513295 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.513525 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.513526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.513526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.513672 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.513727 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.513937 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.514037 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:49.514183 28367 heartbeater.cc:511] Master 127.18.63.125:45135 requested a full tablet report, sending...
I20260808 07:06:49.514461 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.498581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.18.63.66:40221)
0808 07:06:49.498750 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.498761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.499699 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.500046 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.500049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.500530 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.500884 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.500894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.502219 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.502219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.503441 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.503445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.504147 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.504148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.504234 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.508073 (+ 3839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.508079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.509809 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.512874 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.512875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.512875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.513093 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.513390 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.513390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.513391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.513450 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.513781 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.513782 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.513821 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.514231 (+ 410us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.514348 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260808 07:06:49.514786 27724 ts_manager.cc:194] Registered new tserver with Master: aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055)
I20260808 07:06:49.514806 28362 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:49.514849 28362 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.514918 28362 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:49.515074 28363 heartbeater.cc:347] Connected to a master server at 127.18.63.124:39067
I20260808 07:06:49.515169 28363 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.515198 26931 ts_manager.cc:194] Re-registered known tserver with Master: aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055)
I20260808 07:06:49.515272 28363 heartbeater.cc:511] Master 127.18.63.124:39067 requested a full tablet report, sending...
I20260808 07:06:49.515583 26931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:52505
I20260808 07:06:49.515969 27957 ts_manager.cc:194] Registered new tserver with Master: aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055)
I20260808 07:06:49.518988 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:37761
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:42871
--webserver_port=34803
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:49.604140 28373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:49.604357 28373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:49.604380 28373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:49.605623 28373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:49.605667 28373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:49.605693 28373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:49.607210 28373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:37761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=34803
--tserver_master_addrs=127.18.63.126:42871,127.18.63.125:45135,127.18.63.124:39067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:49.607425 28373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:49.607648 28373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:49.612927 28379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:49.612921 28383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.612950 28373 server_base.cc:1034] running on GCE node
W20260808 07:06:49.612922 28380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:49.613425 28373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:49.613685 28373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:49.614847 28373 hybrid_clock.cc:648] HybridClock initialized: now 1786172809614837 us; error 42 us; skew 500 ppm
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:49.617856 28373 init.cc:377] Logged in from keytab as oryx/127.18.63.67@KRBTEST.COM (short username oryx)
I20260808 07:06:49.618233 28373 webserver.cc:460] Webserver started at http://127.18.63.67:34803/ using document root <none> and password file <none>
I20260808 07:06:49.618414 28373 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:49.618466 28373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:49.619701 28373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:49.620275 28394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.620456 28373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:49.620559 28373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "e6d1ce42008c48e3bb5dddf51b495626"
format_stamp: "Formatted at 2026-08-08 07:06:48 on dist-test-slave-bw88"
server_key: "655c4464297d9dc9ef87dd1ed2e0be9f"
server_key_iv: "4ae402d5fd0230360ebd738a50860b8e"
server_key_version: "encryptionkey@0"
I20260808 07:06:49.620905 28373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:49.630155 28373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:49.631843 28373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:49.632004 28373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:49.632220 28373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:49.632529 28373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:49.632593 28373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.632633 28373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:49.632660 28373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:49.638761 28373 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:37761
I20260808 07:06:49.638834 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:37761 every 8 connection(s)
I20260808 07:06:49.639194 28373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:49.644414 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 28373
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.125@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.124@KRBTEST.COM
Aug 08 07:06:49 dist-test-slave-bw88 krb5kdc[26876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172809, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:49.650470 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.639942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.18.63.67:59033)
0808 07:06:49.640214 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.640223 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.640809 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.641043 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.641044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.641597 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.641661 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.641663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642596 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.642597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.643518 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.643521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643969 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.644049 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.644868 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.644872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.646585 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.648618 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.648625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.648628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.648965 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.649202 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.649203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.649203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.649370 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.649731 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.649737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.649907 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.650026 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.650207 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":75,"thread_start_us":117,"threads_started":1}
I20260808 07:06:49.650715 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.640480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:39597 (local address 127.18.63.124:39067)
0808 07:06:49.640527 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.640529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.640537 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:49.640905 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.640906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.640918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.640952 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.641755 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642255 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643577 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643836 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643882 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.647353 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.647354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.647355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.647363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.649468 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.649881 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.649882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.649882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.649940 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650174 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.650175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.650175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.650288 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.650329 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.650523 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.650607 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:49.651028 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.640263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.18.63.67:39597)
0808 07:06:49.640396 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.640423 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.640805 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.641043 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.641044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.641446 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.641635 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.641639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642358 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.642359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.643466 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.643469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643961 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.644017 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.644692 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.644700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.647243 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.649556 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.649557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.649558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.649787 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.650021 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.650022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.650088 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.650376 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.650377 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.650418 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.650825 (+ 407us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.650927 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260808 07:06:49.651193 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.639993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:59033 (local address 127.18.63.125:45135)
0808 07:06:49.640061 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.640064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.640311 (+ 247us) server_negotiation.cc:407] Connection header received
0808 07:06:49.640904 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.640906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.640918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.640952 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.641768 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642506 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643601 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643890 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643967 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.646686 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.646688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.646688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.646698 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.648515 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.649057 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.649059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.649059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.649123 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.649458 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.649459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.649460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.649628 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.649700 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.651008 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.651101 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:49.651309 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.639956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:42553 (local address 127.18.63.126:42871)
0808 07:06:49.640010 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.640013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.640279 (+ 266us) server_negotiation.cc:407] Connection header received
0808 07:06:49.640899 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.640900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.640914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.640964 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.641731 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642511 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643434 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643700 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643766 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.648119 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.648121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.648121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.648129 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.650085 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650460 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.650462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.650462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.650520 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650736 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.650737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.650737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.650877 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.650917 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.651136 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.651215 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:49.651435 28544 heartbeater.cc:347] Connected to a master server at 127.18.63.124:39067
I20260808 07:06:49.651530 28544 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.651609 28545 heartbeater.cc:347] Connected to a master server at 127.18.63.125:45135
I20260808 07:06:49.651664 28545 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.651744 28544 heartbeater.cc:511] Master 127.18.63.124:39067 requested a full tablet report, sending...
I20260808 07:06:49.651744 28545 heartbeater.cc:511] Master 127.18.63.125:45135 requested a full tablet report, sending...
I20260808 07:06:49.651970 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.639941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:42553)
0808 07:06:49.640131 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.640174 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.640786 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.641040 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.641043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.641429 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:49.641632 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.641637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.642625 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.642626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.643339 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.643342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.643785 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.643786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.643943 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.644776 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:49.644782 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:49.647999 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:49.650187 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.650188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.650376 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.650601 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:49.650602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:49.650603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:49.650669 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:49.650961 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:49.650962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:49.651002 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.651419 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.651530 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":59,"thread_start_us":108,"threads_started":1}
I20260808 07:06:49.652187 27724 ts_manager.cc:194] Registered new tserver with Master: e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:49.652187 27957 ts_manager.cc:194] Registered new tserver with Master: e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:49.652300 28542 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42871
I20260808 07:06:49.652340 28542 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:49.652406 28542 heartbeater.cc:511] Master 127.18.63.126:42871 requested a full tablet report, sending...
I20260808 07:06:49.652642 26931 ts_manager.cc:194] Re-registered known tserver with Master: e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:49.653052 26931 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:42553
I20260808 07:06:49.659673 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.648196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.18.63.124:39067)
0808 07:06:49.648663 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.648665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.649127 (+ 462us) server_negotiation.cc:407] Connection header received
0808 07:06:49.649150 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.649151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.649165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.649231 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.650097 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.651016 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.654735 (+ 3719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.654952 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.655046 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.656607 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.656608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.656608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.656615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.658336 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.658697 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.658699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.658699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.658755 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.658964 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.658965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.658966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.659090 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.659204 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.659478 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.659581 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":51,"threads_started":1}
I20260808 07:06:49.660213 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.648452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.18.63.125:45135)
0808 07:06:49.651272 (+ 2820us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.651274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.651281 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:49.651300 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.651300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.651310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.651373 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.653031 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.653653 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.654737 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.655030 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.655093 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.656895 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.656897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.656897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.656904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.658923 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.659317 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.659318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.659318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.659378 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.659620 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.659621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.659621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.659754 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.659846 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.660037 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.660118 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2806,"thread_start_us":186,"threads_started":1}
I20260808 07:06:49.660310 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.647591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.18.63.126:42871)
0808 07:06:49.647756 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.647758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.652552 (+ 4794us) server_negotiation.cc:407] Connection header received
0808 07:06:49.652581 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.652582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.652596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.652671 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.653729 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.654242 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.655233 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.655497 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.655529 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.657085 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.657087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.657087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.657094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.658780 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.659238 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.659240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.659240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.659299 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.659563 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.659564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.659565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.659702 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.659902 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.660100 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.660197 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":108,"threads_started":1}
I20260808 07:06:49.662022 26931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:49.663579 26931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:49.671336 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.667682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46189 (local address 127.0.0.1:58658)
0808 07:06:49.667794 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.667805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.667866 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.668382 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.668383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.668384 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.668422 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.668423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.669466 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.669467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.670386 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.670388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.671074 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.671075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.671134 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.671137 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.671219 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.671264 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 07:06:49.671762 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.667991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.18.63.65:46189)
0808 07:06:49.668188 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.668193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.668204 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:49.668256 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.668259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.668279 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.668386 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.668506 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.669327 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.670493 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.670988 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.671076 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.671166 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.671234 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.671271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260808 07:06:49.672565 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.668199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.18.63.67:37761)
0808 07:06:49.668511 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.668515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.668525 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:49.668562 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.668564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.668579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.668683 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.668806 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.669927 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.671394 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.672186 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.672262 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.672331 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.672428 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.672474 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":99,"threads_started":1}
I20260808 07:06:49.672611 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.667762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:37761 (local address 127.0.0.1:49448)
0808 07:06:49.668143 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.668153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.668204 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.668691 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.668692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.668694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.668735 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.668737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.670056 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.670057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.671288 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.671290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.672275 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.672276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.672340 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.672344 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.672412 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.672454 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":31,"thread_start_us":190,"threads_started":1}
I20260808 07:06:49.672626 28105 tablet_service.cc:1467] Processing CreateTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 (DEFAULT_TABLE table=test-table [id=472e991a669949b98c392d255d6b77ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:49.672827 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.667757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40055 (local address 127.0.0.1:45146)
0808 07:06:49.667971 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.667982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.668043 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.668697 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.668698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.668700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.668749 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.668750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.670093 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.670094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.671288 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.671290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.672572 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.672574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.672648 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.672653 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.672709 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.672758 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":26,"thread_start_us":129,"threads_started":1}
I20260808 07:06:49.673087 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.667943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.18.63.66:40055)
0808 07:06:49.668436 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.668441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.668453 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:49.668499 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.668501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.668520 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.668612 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.668831 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.670008 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.671394 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.672403 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.672466 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.672534 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.672954 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.673008 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":99,"threads_started":1}
I20260808 07:06:49.673089 28105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41017a2498f04bdaa0c81a1b63117fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:49.673367 28455 tablet_service.cc:1467] Processing CreateTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 (DEFAULT_TABLE table=test-table [id=472e991a669949b98c392d255d6b77ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:49.673714 28455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41017a2498f04bdaa0c81a1b63117fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:49.673779 28277 tablet_service.cc:1467] Processing CreateTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 (DEFAULT_TABLE table=test-table [id=472e991a669949b98c392d255d6b77ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:49.674126 28277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41017a2498f04bdaa0c81a1b63117fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:49.676867 28579 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Bootstrap starting.
I20260808 07:06:49.677110 28580 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Bootstrap starting.
I20260808 07:06:49.677281 28578 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Bootstrap starting.
I20260808 07:06:49.677603 28579 tablet_bootstrap.cc:654] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:49.677805 28580 tablet_bootstrap.cc:654] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:49.677942 28579 log.cc:824] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:49.677977 28578 tablet_bootstrap.cc:654] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:49.678153 28580 log.cc:824] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:49.678297 28578 log.cc:824] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:49.679042 28579 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: No bootstrap required, opened a new log
I20260808 07:06:49.679133 28579 ts_tablet_manager.cc:1397] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:49.679215 28578 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: No bootstrap required, opened a new log
I20260808 07:06:49.679283 28578 ts_tablet_manager.cc:1397] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:49.679421 28580 tablet_bootstrap.cc:492] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: No bootstrap required, opened a new log
I20260808 07:06:49.679493 28580 ts_tablet_manager.cc:1397] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:49.681141 28579 raft_consensus.cc:348] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.681358 28579 raft_consensus.cc:374] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:49.681404 28579 raft_consensus.cc:729] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42095ed2821f4ddd9b3281ff509327a1, State: Initialized, Role: FOLLOWER
I20260808 07:06:49.681288 28578 raft_consensus.cc:348] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.681519 28579 consensus_queue.cc:260] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.681458 28580 raft_consensus.cc:348] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.681602 28580 raft_consensus.cc:374] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:49.681663 28580 raft_consensus.cc:729] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d1ce42008c48e3bb5dddf51b495626, State: Initialized, Role: FOLLOWER
I20260808 07:06:49.681506 28578 raft_consensus.cc:374] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:49.681718 28578 raft_consensus.cc:729] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff17fccf9fb401b8390ddf41368827c, State: Initialized, Role: FOLLOWER
I20260808 07:06:49.681795 28579 ts_tablet_manager.cc:1428] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:49.681804 28580 consensus_queue.cc:260] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.681880 28578 consensus_queue.cc:260] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.682013 28183 heartbeater.cc:503] Master 127.18.63.126:42871 was elected leader, sending a full tablet report...
I20260808 07:06:49.682106 28580 ts_tablet_manager.cc:1428] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:49.682199 28578 ts_tablet_manager.cc:1428] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:49.682255 28542 heartbeater.cc:503] Master 127.18.63.126:42871 was elected leader, sending a full tablet report...
I20260808 07:06:49.682413 28362 heartbeater.cc:503] Master 127.18.63.126:42871 was elected leader, sending a full tablet report...
I20260808 07:06:49.718606 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.716028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.18.63.124:39067)
0808 07:06:49.716101 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.716104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.716112 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:49.716145 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.716145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.716168 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.716241 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.716396 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.716897 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.717844 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.718260 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.718355 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.718445 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.718493 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.718538 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:06:49.718606 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.715924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39478)
0808 07:06:49.715984 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.715997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.716059 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.716244 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.716245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.716246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.716302 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.716304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.716981 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.716982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.717746 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.717749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.718358 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.718358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.718421 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.718424 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.718476 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.718524 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:06:49.719038 27971 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260808 07:06:49.720014 28011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928f91425d204020b989476350668cf6. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.720139 28011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: LEARNER
W20260808 07:06:49.748530 28368 tablet.cc:2369] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:49.795254 28588 raft_consensus.cc:482] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:49.795385 28588 raft_consensus.cc:504] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.795812 28588 leader_election.cc:288] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189), e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:49.798699 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.796083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:44977 (local address 127.18.63.65:46189)
0808 07:06:49.796140 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.796143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.796152 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:49.796212 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.796213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.796229 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.796297 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.796462 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.796943 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.797986 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.798347 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.798433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.798442 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.798589 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.798628 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:06:49.798700 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.796004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46189 (local address 127.18.63.66:44977)
0808 07:06:49.796089 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.796102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.796149 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.796302 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.796303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.796305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.796386 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.796387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.797037 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.797038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.797892 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.797894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.798431 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.798432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.798504 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.798507 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.798580 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.798628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 07:06:49.798940 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.796015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:37761 (local address 127.18.63.66:42957)
0808 07:06:49.796110 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:49.796123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:49.796184 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:49.796347 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:49.796348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:49.796350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:49.796429 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.796430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.797165 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.797167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.798053 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:49.798055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.798627 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.798628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:49.798718 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.798721 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:49.798793 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:06:49.798844 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":4}
I20260808 07:06:49.799058 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.796080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:42957 (local address 127.18.63.67:37761)
0808 07:06:49.796133 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.796135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.796163 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:06:49.796255 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.796256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.796271 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.796342 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:49.796529 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.797053 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.798153 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.798531 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.798627 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.798636 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.798922 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.798977 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:06:49.799175 28130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41017a2498f04bdaa0c81a1b63117fa9" candidate_uuid: "aff17fccf9fb401b8390ddf41368827c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42095ed2821f4ddd9b3281ff509327a1" is_pre_election: true
I20260808 07:06:49.799431 28130 raft_consensus.cc:2454] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff17fccf9fb401b8390ddf41368827c in term 0.
I20260808 07:06:49.799396 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41017a2498f04bdaa0c81a1b63117fa9" candidate_uuid: "aff17fccf9fb401b8390ddf41368827c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d1ce42008c48e3bb5dddf51b495626" is_pre_election: true
I20260808 07:06:49.799549 28480 raft_consensus.cc:2454] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff17fccf9fb401b8390ddf41368827c in term 0.
I20260808 07:06:49.799692 28217 leader_election.cc:302] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42095ed2821f4ddd9b3281ff509327a1, aff17fccf9fb401b8390ddf41368827c; no voters:
I20260808 07:06:49.799824 28588 raft_consensus.cc:2781] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:49.799889 28588 raft_consensus.cc:482] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:49.799907 28588 raft_consensus.cc:3037] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:49.800652 28588 raft_consensus.cc:504] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.800802 28588 leader_election.cc:288] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [CANDIDATE]: Term 1 election: Requested vote from peers 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189), e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761)
I20260808 07:06:49.801023 28130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41017a2498f04bdaa0c81a1b63117fa9" candidate_uuid: "aff17fccf9fb401b8390ddf41368827c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42095ed2821f4ddd9b3281ff509327a1"
I20260808 07:06:49.801024 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41017a2498f04bdaa0c81a1b63117fa9" candidate_uuid: "aff17fccf9fb401b8390ddf41368827c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d1ce42008c48e3bb5dddf51b495626"
I20260808 07:06:49.801116 28130 raft_consensus.cc:3037] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:49.801116 28480 raft_consensus.cc:3037] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:49.802675 28480 raft_consensus.cc:2454] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff17fccf9fb401b8390ddf41368827c in term 1.
I20260808 07:06:49.802675 28130 raft_consensus.cc:2454] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff17fccf9fb401b8390ddf41368827c in term 1.
I20260808 07:06:49.802886 28217 leader_election.cc:302] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42095ed2821f4ddd9b3281ff509327a1, aff17fccf9fb401b8390ddf41368827c; no voters:
I20260808 07:06:49.803011 28588 raft_consensus.cc:2781] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:49.803201 28588 raft_consensus.cc:686] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 1 LEADER]: Becoming Leader. State: Replica: aff17fccf9fb401b8390ddf41368827c, State: Running, Role: LEADER
I20260808 07:06:49.803304 28588 consensus_queue.cc:237] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } }
I20260808 07:06:49.804150 26930 catalog_manager.cc:5495] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c reported cstate change: term changed from 0 to 1, leader changed from <none> to aff17fccf9fb401b8390ddf41368827c (127.18.63.66). New cstate: current_term: 1 leader_uuid: "aff17fccf9fb401b8390ddf41368827c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:49.804795 27914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: NON_VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: true } } }
I20260808 07:06:49.804899 27914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.805462 27914 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: attempting to promote NON_VOTER b1a4105367784ede815cd51cba247166 to VOTER
W20260808 07:06:49.805600 27676 consensus_queue.cc:1159] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" responder_term: 1 status { last_received { term: 1 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 1 index: 10 } }
I20260808 07:06:49.805739 27914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:49.806183 27971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 LEARNER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 07:06:49.806183 27737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Refusing update from remote peer 928f91425d204020b989476350668cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 07:06:49.806473 27677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260808 07:06:49.806581 27914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260808 07:06:49.807273 27676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b1a4105367784ede815cd51cba247166 (127.18.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.807488 27971 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b1a4105367784ede815cd51cba247166 (127.18.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.808295 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.808378 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.808334 27677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.808418 27677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.808439 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.808506 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:49.808670 28011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.808792 28011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:49.808781 27737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b1a4105367784ede815cd51cba247166 (127.18.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.809818 28597 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928f91425d204020b989476350668cf6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:49.809932 28597 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:49.816388 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.813973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.18.63.66:40055)
0808 07:06:49.814027 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.814029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.814037 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:49.814130 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.814131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.814150 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.814237 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:49.814355 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.814827 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.815559 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.815788 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.815866 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.816039 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:49.816152 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:49.816240 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.816282 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.816303 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:06:49.818197 28480 raft_consensus.cc:1261] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Refusing update from remote peer aff17fccf9fb401b8390ddf41368827c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:49.818197 28130 raft_consensus.cc:1261] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Refusing update from remote peer aff17fccf9fb401b8390ddf41368827c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:49.818542 28588 consensus_queue.cc:1035] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [LEADER]: Connected to new peer: Peer: permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:49.818727 28594 consensus_queue.cc:1035] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:49.821772 28602 mvcc.cc:205] Tried to move back new op lower bound from 7316163829012537344 to 7316163828954849280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163829012537344}]
I20260808 07:06:49.822674 28600 mvcc.cc:205] Tried to move back new op lower bound from 7316163829012537344 to 7316163828954849280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163829012537344}]
I20260808 07:06:49.823647 28601 mvcc.cc:205] Tried to move back new op lower bound from 7316163829012537344 to 7316163828954849280. Current Snapshot: MvccSnapshot[applied={T|T < 7316163829012537344 or (T in {7316163829012537344})}]
I20260808 07:06:49.824919 26932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:49.825007 26932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:49.826766 26932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 928f91425d204020b989476350668cf6: Sending DeleteTablet for 3 replicas of tablet 41017a2498f04bdaa0c81a1b63117fa9
I20260808 07:06:49.827065 28105 tablet_service.cc:1514] Processing DeleteTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:49 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:58658
I20260808 07:06:49.827169 28277 tablet_service.cc:1514] Processing DeleteTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:49 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:45146
I20260808 07:06:49.827453 28455 tablet_service.cc:1514] Processing DeleteTablet for tablet 41017a2498f04bdaa0c81a1b63117fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:49 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49448
I20260808 07:06:49.827697 28616 tablet_replica.cc:330] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: stopping tablet replica
I20260808 07:06:49.827700 28612 tablet_replica.cc:330] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: stopping tablet replica
I20260808 07:06:49.827807 28612 raft_consensus.cc:2229] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:49.827895 28616 raft_consensus.cc:2229] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:49.827919 28612 raft_consensus.cc:2258] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:49.828049 28616 raft_consensus.cc:2258] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:49.828169 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 26891
I20260808 07:06:49.832377 28612 ts_tablet_manager.cc:1905] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:49.834949 28612 ts_tablet_manager.cc:1918] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:49.835047 28612 log.cc:1196] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/41017a2498f04bdaa0c81a1b63117fa9
I20260808 07:06:49.835425 28612 ts_tablet_manager.cc:1939] T 41017a2498f04bdaa0c81a1b63117fa9 P 42095ed2821f4ddd9b3281ff509327a1: Deleting consensus metadata
I20260808 07:06:49.836227 28616 ts_tablet_manager.cc:1905] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:49.837966 28616 ts_tablet_manager.cc:1918] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:49.837996 28614 tablet_replica.cc:330] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: stopping tablet replica
I20260808 07:06:49.838133 28616 log.cc:1196] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/41017a2498f04bdaa0c81a1b63117fa9
I20260808 07:06:49.838135 28614 raft_consensus.cc:2229] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:49.838279 28614 raft_consensus.cc:2258] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:49.838521 28616 ts_tablet_manager.cc:1939] T 41017a2498f04bdaa0c81a1b63117fa9 P e6d1ce42008c48e3bb5dddf51b495626: Deleting consensus metadata
I20260808 07:06:49.841825 28614 ts_tablet_manager.cc:1905] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:49.843447 28614 ts_tablet_manager.cc:1918] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:49.843585 28614 log.cc:1196] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/41017a2498f04bdaa0c81a1b63117fa9
I20260808 07:06:49.843930 28614 ts_tablet_manager.cc:1939] T 41017a2498f04bdaa0c81a1b63117fa9 P aff17fccf9fb401b8390ddf41368827c: Deleting consensus metadata
W20260808 07:06:49.844949 28214 connection.cc:535] server connection from 127.0.0.1:45146 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:06:49.845150 28046 connection.cc:535] server connection from 127.0.0.1:58658 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:06:49.845412 28402 connection.cc:535] server connection from 127.0.0.1:49448 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:06:49.850513 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.827679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39488 (local address 127.18.63.124:39067)
0808 07:06:49.827721 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.827723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.841480 (+ 13757us) server_negotiation.cc:407] Connection header received
0808 07:06:49.841520 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.841521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.841536 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.841595 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.842795 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.843620 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.844758 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.845082 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.845156 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.847287 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.847289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.847289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.847297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.849235 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.849657 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.849658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.849659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.849714 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.849916 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.849916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.849917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.850041 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.850135 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.850333 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.850415 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:49.855361 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.846934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.18.63.124:39067)
0808 07:06:49.846975 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.846977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.847203 (+ 226us) server_negotiation.cc:407] Connection header received
0808 07:06:49.847303 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.847304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.847315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.847345 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.849062 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.849847 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.850894 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.851165 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.851206 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.852475 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.852476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.852476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.852483 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.854056 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.854442 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.854443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.854444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.854500 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.854751 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.854753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.854753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.854888 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.854986 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.855184 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.855264 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:49.855491 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:49.846903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.18.63.125:45135)
0808 07:06:49.846951 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:49.846953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:49.847150 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:06:49.847263 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:49.847264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:49.847276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:49.847308 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:49.848494 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.849269 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.850415 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:49.850732 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:49.850772 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:49.852725 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:49.852726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:49.852726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:49.852734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:49.854300 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.854679 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.854681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.854681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.854741 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:49.854920 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:49.854921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:49.854921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:49.855041 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:49.855192 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:49.855294 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 07:06:49.855366 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:50.823271 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:50.822940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:36125)
0808 07:06:50.823100 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:50.823212 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
W20260808 07:06:50.823582 28542 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:42871 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:50.823792 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:50.823487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:36709)
0808 07:06:50.823624 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:50.823739 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260808 07:06:50.824187 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:50.823915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:56807)
0808 07:06:50.824046 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:50.824142 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
W20260808 07:06:50.824193 28362 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:42871 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
W20260808 07:06:50.824388 28183 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:42871 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.327242 28631 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 928f91425d204020b989476350668cf6)
I20260808 07:06:51.327387 28632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 928f91425d204020b989476350668cf6)
I20260808 07:06:51.327430 28631 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:51.327556 28632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:51.327757 28631 leader_election.cc:288] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), b1a4105367784ede815cd51cba247166 (127.18.63.124:39067)
I20260808 07:06:51.327921 28632 leader_election.cc:288] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125:45135)
I20260808 07:06:51.328430 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.328130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37124)
0808 07:06:51.328326 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.328381 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
W20260808 07:06:51.328553 27948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.328648 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.327903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37112)
0808 07:06:51.328553 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.328612 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":66,"threads_started":1}
W20260808 07:06:51.328768 27706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.329353 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.329208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37136)
0808 07:06:51.329265 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.329296 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 07:06:51.329438 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.329315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37150)
0808 07:06:51.329376 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.329403 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 07:06:51.329445 27706 leader_election.cc:334] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
W20260808 07:06:51.329528 27948 leader_election.cc:334] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.331758 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.328232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.18.63.124:39067)
0808 07:06:51.328811 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:51.328813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:51.329106 (+ 293us) server_negotiation.cc:407] Connection header received
0808 07:06:51.329289 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:51.329290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:51.329300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:51.329400 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:51.329573 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.330057 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331007 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.331392 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:51.331486 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:51.331633 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:06:51.331677 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":52,"threads_started":1}
I20260808 07:06:51.331784 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.328321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39067 (local address 127.0.0.1:39502)
0808 07:06:51.329030 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.329041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:51.329120 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:51.329459 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:51.329460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:51.329462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:51.329510 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:51.329512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.330162 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.330163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:51.330915 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:51.330917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.331475 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:51.331539 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:51.331543 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:51.331632 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:06:51.331677 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":35,"threads_started":1}
I20260808 07:06:51.331770 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.328133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45135 (local address 127.0.0.1:41192)
0808 07:06:51.328398 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.328408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:51.328468 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:51.329127 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:51.329128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:51.329131 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:51.329198 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:51.329201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.330161 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.330162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:51.330915 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:51.330917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.331480 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:51.331552 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:51.331555 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:51.331636 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:06:51.331681 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":44,"threads_started":1}
I20260808 07:06:51.331784 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.328223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.18.63.125:45135)
0808 07:06:51.328663 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:51.328666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:51.328802 (+ 136us) server_negotiation.cc:407] Connection header received
0808 07:06:51.328922 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:51.328923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:51.328937 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:51.329039 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:51.329291 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.330060 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331020 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:51.331399 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:51.331480 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:51.331490 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:51.331645 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:06:51.331694 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":243,"thread_start_us":66,"threads_started":1}
I20260808 07:06:51.332055 27971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b1a4105367784ede815cd51cba247166" is_pre_election: true
I20260808 07:06:51.332190 27971 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8ed3e8eccbd4deaacd3b51f30660bdd in term 1.
I20260808 07:06:51.332180 27737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1a4105367784ede815cd51cba247166" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" is_pre_election: true
I20260808 07:06:51.332273 27737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a4105367784ede815cd51cba247166 in term 1.
I20260808 07:06:51.332345 27706 leader_election.cc:302] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b1a4105367784ede815cd51cba247166, d8ed3e8eccbd4deaacd3b51f30660bdd; no voters: 928f91425d204020b989476350668cf6
I20260808 07:06:51.332422 27947 leader_election.cc:302] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b1a4105367784ede815cd51cba247166, d8ed3e8eccbd4deaacd3b51f30660bdd; no voters: 928f91425d204020b989476350668cf6
I20260808 07:06:51.332448 28631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:06:51.332506 28631 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 928f91425d204020b989476350668cf6)
I20260808 07:06:51.332525 28632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 07:06:51.332537 28631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:06:51.332564 28632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 928f91425d204020b989476350668cf6)
I20260808 07:06:51.332598 28632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:06:51.333281 28631 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:51.333288 28632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:51.333442 28631 leader_election.cc:288] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 election: Requested vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), b1a4105367784ede815cd51cba247166 (127.18.63.124:39067)
I20260808 07:06:51.333449 28632 leader_election.cc:288] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 election: Requested vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125:45135)
I20260808 07:06:51.333659 27737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1a4105367784ede815cd51cba247166" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
I20260808 07:06:51.333741 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.333621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37172)
0808 07:06:51.333683 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.333716 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:06:51.333755 27737 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b1a4105367784ede815cd51cba247166 in current term 2: Already voted for candidate d8ed3e8eccbd4deaacd3b51f30660bdd in this term.
I20260808 07:06:51.333788 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.333524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37160)
0808 07:06:51.333711 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.333758 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260808 07:06:51.333905 27971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b1a4105367784ede815cd51cba247166"
I20260808 07:06:51.334019 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.333926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37180)
0808 07:06:51.333978 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.334000 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:06:51.334025 27971 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d8ed3e8eccbd4deaacd3b51f30660bdd in current term 2: Already voted for candidate b1a4105367784ede815cd51cba247166 in this term.
W20260808 07:06:51.334072 27706 leader_election.cc:334] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.334144 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.333971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:37188)
0808 07:06:51.334010 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.334031 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 07:06:51.334203 27948 leader_election.cc:334] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:51.334208 27706 leader_election.cc:302] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8ed3e8eccbd4deaacd3b51f30660bdd; no voters: 928f91425d204020b989476350668cf6, b1a4105367784ede815cd51cba247166
I20260808 07:06:51.334244 27948 leader_election.cc:302] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1a4105367784ede815cd51cba247166; no voters: 928f91425d204020b989476350668cf6, d8ed3e8eccbd4deaacd3b51f30660bdd
I20260808 07:06:51.334313 28631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 07:06:51.334304 28632 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 07:06:51.824219 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.823971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:49677)
0808 07:06:51.824110 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.824157 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260808 07:06:51.824884 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.824646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:37843)
0808 07:06:51.824796 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.824835 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260808 07:06:51.824919 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:51.824657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:56053)
0808 07:06:51.824839 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:51.824876 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260808 07:06:52.825103 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:52.824776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:47503)
0808 07:06:52.824966 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:52.825014 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 07:06:52.825640 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:52.825315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:49577)
0808 07:06:52.825555 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:52.825595 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":72,"threads_started":1}
I20260808 07:06:52.825693 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:52.825400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:33715)
0808 07:06:52.825605 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:52.825649 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260808 07:06:53.162688 28654 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:53.162823 28654 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:53.163049 28654 leader_election.cc:288] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125:45135)
I20260808 07:06:53.163312 27737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1a4105367784ede815cd51cba247166" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" is_pre_election: true
I20260808 07:06:53.163427 27737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1a4105367784ede815cd51cba247166 in term 2.
I20260808 07:06:53.163417 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.163211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55518)
0808 07:06:53.163333 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.163376 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260808 07:06:53.163618 27947 leader_election.cc:302] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1a4105367784ede815cd51cba247166, d8ed3e8eccbd4deaacd3b51f30660bdd; no voters:
I20260808 07:06:53.163726 28654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 07:06:53.163769 28654 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:53.163801 28654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:06:53.164004 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.163896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55532)
0808 07:06:53.163949 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.163983 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:06:53.164085 27948 leader_election.cc:334] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:53.164791 28654 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:53.164942 28654 leader_election.cc:288] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 election: Requested vote from peers 928f91425d204020b989476350668cf6 (127.18.63.126:42871), d8ed3e8eccbd4deaacd3b51f30660bdd (127.18.63.125:45135)
I20260808 07:06:53.165120 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.165018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55546)
0808 07:06:53.165054 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.165087 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:06:53.165139 27737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1a4105367784ede815cd51cba247166" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd"
I20260808 07:06:53.165254 27737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:06:53.165511 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.165406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55552)
0808 07:06:53.165470 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.165495 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 07:06:53.165613 27948 leader_election.cc:334] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
I20260808 07:06:53.166126 27737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1a4105367784ede815cd51cba247166 in term 3.
I20260808 07:06:53.166302 27947 leader_election.cc:302] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b1a4105367784ede815cd51cba247166, d8ed3e8eccbd4deaacd3b51f30660bdd; no voters: 928f91425d204020b989476350668cf6
I20260808 07:06:53.166402 28654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:06:53.166510 28654 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [term 3 LEADER]: Becoming Leader. State: Replica: b1a4105367784ede815cd51cba247166, State: Running, Role: LEADER
I20260808 07:06:53.166610 28654 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } }
I20260808 07:06:53.166996 28657 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a4105367784ede815cd51cba247166. Latest consensus state: current_term: 3 leader_uuid: "b1a4105367784ede815cd51cba247166" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:53.167109 28657 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:53.167320 28659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:53.167768 28659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:53.167832 28659 catalog_manager.cc:1259] Loaded cluster ID: 47124f5937ad452c84cf1f8f5c4f010f
I20260808 07:06:53.167848 28659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:53.167997 28659 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:53.168048 28659 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Loaded TSK: 0
I20260808 07:06:53.168143 28659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:06:53.184180 27957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:53.185956 27957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:53.187075 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.186936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55568)
0808 07:06:53.187008 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.187045 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 07:06:53.187063 27737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [term 3 FOLLOWER]: Refusing update from remote peer b1a4105367784ede815cd51cba247166: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 07:06:53.187296 28657 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 07:06:53.187446 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.187341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.0.0.1:55580)
0808 07:06:53.187394 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.187425 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:06:53.187999 27948 consensus_peers.cc:487] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 -> Peer 928f91425d204020b989476350668cf6 (127.18.63.126:42871): Couldn't send request to peer 928f91425d204020b989476350668cf6. Status: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 07:06:53.188071 28661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a4105367784ede815cd51cba247166. Latest consensus state: current_term: 3 leader_uuid: "b1a4105367784ede815cd51cba247166" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:53.188185 28661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:53.188582 28660 mvcc.cc:205] Tried to move back new op lower bound from 7316163842811863040 to 7316163842731044864. Current Snapshot: MvccSnapshot[applied={T|T < 7316163842811863040}]
I20260808 07:06:53.188691 28657 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b1a4105367784ede815cd51cba247166" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:53.188755 28657 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:53.188823 28661 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b1a4105367784ede815cd51cba247166" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "928f91425d204020b989476350668cf6" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42871 } } peers { permanent_uuid: "d8ed3e8eccbd4deaacd3b51f30660bdd" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45135 } attrs { promote: false } } peers { permanent_uuid: "b1a4105367784ede815cd51cba247166" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39067 } attrs { promote: false } } }
I20260808 07:06:53.188918 28661 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ed3e8eccbd4deaacd3b51f30660bdd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:06:53.188961 28662 mvcc.cc:205] Tried to move back new op lower bound from 7316163842811863040 to 7316163842731044864. Current Snapshot: MvccSnapshot[applied={T|T < 7316163842811863040}]
I20260808 07:06:53.194159 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46189 (local address 127.0.0.1:35422)
0808 07:06:53.190722 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.190732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:53.190784 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:53.191078 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:53.191079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:53.191081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:53.191141 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.191142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.191858 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.191859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.193104 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.193107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.193903 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.193968 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.193973 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:53.194031 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 07:06:53.194074 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":48}
I20260808 07:06:53.194169 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.18.63.65:46189)
0808 07:06:53.190948 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.190950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.190959 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:53.190985 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.190985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.190995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.191074 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:53.191235 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.191776 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193209 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.193823 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193898 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.193915 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.194053 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.194098 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260808 07:06:53.194175 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:37761 (local address 127.0.0.1:59792)
0808 07:06:53.190826 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.190836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:53.190909 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:53.191209 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:53.191210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:53.191212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:53.191261 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.191263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.191987 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.191988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.193112 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.193114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.193924 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.193992 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.193995 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:53.194047 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:06:53.194090 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260808 07:06:53.194463 28105 tablet_service.cc:1467] Processing CreateTablet for tablet bbecaf007ec34c9296b7e56fec98946d (DEFAULT_TABLE table=test-table [id=9284d01a59bb4f59b342327441dc0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:53.194614 28105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbecaf007ec34c9296b7e56fec98946d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:53.194814 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40055 (local address 127.0.0.1:38480)
0808 07:06:53.191272 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.191281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:53.191329 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:53.192004 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:53.192005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:53.192008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:53.192049 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.192051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.192658 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.192659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.193635 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.193637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.194573 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.194574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.194637 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.194642 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:53.194691 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:06:53.194729 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":485,"thread_start_us":82,"threads_started":1}
I20260808 07:06:53.194841 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.18.63.67:37761)
0808 07:06:53.191060 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.191062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.191069 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:53.191094 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.191095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.191113 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.191330 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:53.191373 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.191877 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193213 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.193819 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.194637 (+ 818us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.194680 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.194703 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.194754 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260808 07:06:53.195027 28455 tablet_service.cc:1467] Processing CreateTablet for tablet bbecaf007ec34c9296b7e56fec98946d (DEFAULT_TABLE table=test-table [id=9284d01a59bb4f59b342327441dc0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:53.195111 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.190871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.18.63.66:40055)
0808 07:06:53.191011 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.191013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.191848 (+ 835us) server_negotiation.cc:407] Connection header received
0808 07:06:53.191876 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.191877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.191886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.192000 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:53.192107 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.192574 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.193725 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.194479 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.194572 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.194586 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.194983 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.195031 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260808 07:06:53.195145 28455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbecaf007ec34c9296b7e56fec98946d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:53.195366 28277 tablet_service.cc:1467] Processing CreateTablet for tablet bbecaf007ec34c9296b7e56fec98946d (DEFAULT_TABLE table=test-table [id=9284d01a59bb4f59b342327441dc0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:53.195514 28277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbecaf007ec34c9296b7e56fec98946d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:53.196130 28678 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1: Bootstrap starting.
I20260808 07:06:53.196475 28678 tablet_bootstrap.cc:654] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:53.196679 28679 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Bootstrap starting.
I20260808 07:06:53.196866 28682 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Bootstrap starting.
I20260808 07:06:53.197099 28679 tablet_bootstrap.cc:654] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:53.197264 28682 tablet_bootstrap.cc:654] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:53.197979 28679 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: No bootstrap required, opened a new log
I20260808 07:06:53.198019 28682 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: No bootstrap required, opened a new log
I20260808 07:06:53.198040 28679 ts_tablet_manager.cc:1397] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:53.198060 28682 ts_tablet_manager.cc:1397] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:53.198066 28678 tablet_bootstrap.cc:492] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1: No bootstrap required, opened a new log
I20260808 07:06:53.198127 28678 ts_tablet_manager.cc:1397] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:53.198180 28679 raft_consensus.cc:348] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198261 28679 raft_consensus.cc:374] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:53.198287 28679 raft_consensus.cc:729] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d1ce42008c48e3bb5dddf51b495626, State: Initialized, Role: FOLLOWER
I20260808 07:06:53.198331 28679 consensus_queue.cc:260] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198203 28682 raft_consensus.cc:348] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198482 28682 raft_consensus.cc:374] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:53.198513 28682 raft_consensus.cc:729] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff17fccf9fb401b8390ddf41368827c, State: Initialized, Role: FOLLOWER
I20260808 07:06:53.198546 28679 ts_tablet_manager.cc:1428] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:53.198509 28678 raft_consensus.cc:348] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198589 28678 raft_consensus.cc:374] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:53.198617 28678 raft_consensus.cc:729] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42095ed2821f4ddd9b3281ff509327a1, State: Initialized, Role: FOLLOWER
I20260808 07:06:53.198576 28682 consensus_queue.cc:260] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198674 28678 consensus_queue.cc:260] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.198855 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.198757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:54921)
0808 07:06:53.198797 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.198828 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 07:06:53.198858 28678 ts_tablet_manager.cc:1428] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:53.199048 28682 ts_tablet_manager.cc:1428] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:53.199272 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.199177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:58469)
0808 07:06:53.199221 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.199246 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 07:06:53.199348 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.199257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:55093)
0808 07:06:53.199299 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.199324 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:06:53.199834 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.199738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:33945)
0808 07:06:53.199777 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.199801 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 07:06:53.199882 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.199786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:41399)
0808 07:06:53.199827 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.199852 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:06:53.199945 28363 heartbeater.cc:503] Master 127.18.63.124:39067 was elected leader, sending a full tablet report...
I20260808 07:06:53.200109 28544 heartbeater.cc:503] Master 127.18.63.124:39067 was elected leader, sending a full tablet report...
I20260808 07:06:53.200744 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.200635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:44873)
0808 07:06:53.200688 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.200716 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:06:53.201043 28184 heartbeater.cc:503] Master 127.18.63.124:39067 was elected leader, sending a full tablet report...
I20260808 07:06:53.225896 28685 raft_consensus.cc:482] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:53.226068 28685 raft_consensus.cc:504] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.226524 28685 leader_election.cc:288] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055), 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
I20260808 07:06:53.230238 28672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.226915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52515 (local address 127.18.63.65:46189)
0808 07:06:53.227021 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.227023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.227032 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:53.227057 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.227058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.227068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.227121 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:53.227388 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.227923 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.229518 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.229879 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.229972 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.229983 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.230132 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.230169 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 07:06:53.230238 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.226693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46189 (local address 127.18.63.67:52515)
0808 07:06:53.226759 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.226770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:53.226830 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:53.227139 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:53.227140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:53.227141 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:53.227216 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.227263 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.228010 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.228011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.229410 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.229412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.229975 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.229976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.230043 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.230047 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:53.230123 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:06:53.230167 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":51}
I20260808 07:06:53.230391 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.226814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:36503 (local address 127.18.63.66:40055)
0808 07:06:53.226864 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.226866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.227012 (+ 146us) server_negotiation.cc:407] Connection header received
0808 07:06:53.227048 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.227049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.227060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.227122 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:53.227539 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.228062 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.229509 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.229914 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.230055 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.230069 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.230273 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.230322 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:53.230402 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.226762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40055 (local address 127.18.63.67:36503)
0808 07:06:53.226935 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.226944 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:53.226999 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:53.227396 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:53.227397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:53.227399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:53.227444 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.227445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.228152 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.228153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.229401 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:53.229405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.230079 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.230081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:53.230172 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.230175 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:53.230262 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 07:06:53.230310 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":126,"threads_started":1}
I20260808 07:06:53.230692 28130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbecaf007ec34c9296b7e56fec98946d" candidate_uuid: "e6d1ce42008c48e3bb5dddf51b495626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42095ed2821f4ddd9b3281ff509327a1" is_pre_election: true
I20260808 07:06:53.230676 28304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbecaf007ec34c9296b7e56fec98946d" candidate_uuid: "e6d1ce42008c48e3bb5dddf51b495626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff17fccf9fb401b8390ddf41368827c" is_pre_election: true
I20260808 07:06:53.230818 28130 raft_consensus.cc:2454] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d1ce42008c48e3bb5dddf51b495626 in term 0.
I20260808 07:06:53.230830 28304 raft_consensus.cc:2454] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d1ce42008c48e3bb5dddf51b495626 in term 0.
I20260808 07:06:53.231033 28401 leader_election.cc:302] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42095ed2821f4ddd9b3281ff509327a1, e6d1ce42008c48e3bb5dddf51b495626; no voters:
I20260808 07:06:53.231211 28685 raft_consensus.cc:2781] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:53.231264 28685 raft_consensus.cc:482] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:53.231280 28685 raft_consensus.cc:3037] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:53.232062 28685 raft_consensus.cc:504] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.232214 28685 leader_election.cc:288] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [CANDIDATE]: Term 1 election: Requested vote from peers aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055), 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189)
I20260808 07:06:53.232383 28304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbecaf007ec34c9296b7e56fec98946d" candidate_uuid: "e6d1ce42008c48e3bb5dddf51b495626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff17fccf9fb401b8390ddf41368827c"
I20260808 07:06:53.232476 28304 raft_consensus.cc:3037] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:53.232463 28130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbecaf007ec34c9296b7e56fec98946d" candidate_uuid: "e6d1ce42008c48e3bb5dddf51b495626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42095ed2821f4ddd9b3281ff509327a1"
I20260808 07:06:53.232522 28130 raft_consensus.cc:3037] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:53.233191 28130 raft_consensus.cc:2454] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d1ce42008c48e3bb5dddf51b495626 in term 1.
I20260808 07:06:53.233223 28304 raft_consensus.cc:2454] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d1ce42008c48e3bb5dddf51b495626 in term 1.
I20260808 07:06:53.233392 28401 leader_election.cc:302] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42095ed2821f4ddd9b3281ff509327a1, e6d1ce42008c48e3bb5dddf51b495626; no voters:
I20260808 07:06:53.233500 28685 raft_consensus.cc:2781] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:53.233664 28685 raft_consensus.cc:686] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 1 LEADER]: Becoming Leader. State: Replica: e6d1ce42008c48e3bb5dddf51b495626, State: Running, Role: LEADER
I20260808 07:06:53.233727 28685 consensus_queue.cc:237] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } }
I20260808 07:06:53.234016 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.233908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:52517)
0808 07:06:53.233958 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.233989 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:06:53.234642 27963 catalog_manager.cc:5495] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "e6d1ce42008c48e3bb5dddf51b495626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d1ce42008c48e3bb5dddf51b495626" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 37761 } health_report { overall_health: HEALTHY } } }
I20260808 07:06:53.240705 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.238674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.18.63.67:37761)
0808 07:06:53.238727 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:53.238728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:53.238736 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:53.238770 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:53.238771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:53.238785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:53.238841 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:53.238931 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.239376 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.240101 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:53.240269 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:53.240316 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:53.240442 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:53.240530 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:53.240582 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:53.240623 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:06:53.240639 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:06:53.241863 28130 raft_consensus.cc:1261] T bbecaf007ec34c9296b7e56fec98946d P 42095ed2821f4ddd9b3281ff509327a1 [term 1 FOLLOWER]: Refusing update from remote peer e6d1ce42008c48e3bb5dddf51b495626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:53.241888 28304 raft_consensus.cc:1261] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Refusing update from remote peer e6d1ce42008c48e3bb5dddf51b495626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:53.242152 28685 consensus_queue.cc:1035] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42095ed2821f4ddd9b3281ff509327a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:53.242280 28689 consensus_queue.cc:1035] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aff17fccf9fb401b8390ddf41368827c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:53.243081 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.242978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:38817)
0808 07:06:53.243022 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.243049 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 07:06:53.243172 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.243079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:48919)
0808 07:06:53.243114 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.243144 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:06:53.244784 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.244646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.65:53359)
0808 07:06:53.244726 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.244753 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 07:06:53.245291 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.244646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:44979)
0808 07:06:53.245231 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.245262 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567}
I20260808 07:06:53.246089 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.245967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.66:44683)
0808 07:06:53.246011 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.246053 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:06:53.246261 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.245733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42871 (local address 127.18.63.67:53571)
0808 07:06:53.246190 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.246220 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:42871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443}
I20260808 07:06:53.247195 27957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:53.247296 27957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:53.248605 27957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1a4105367784ede815cd51cba247166: Sending DeleteTablet for 3 replicas of tablet bbecaf007ec34c9296b7e56fec98946d
I20260808 07:06:53.248912 28277 tablet_service.cc:1514] Processing DeleteTablet for tablet bbecaf007ec34c9296b7e56fec98946d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:53 UTC) from {username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:38480
I20260808 07:06:53.248970 28105 tablet_service.cc:1514] Processing DeleteTablet for tablet bbecaf007ec34c9296b7e56fec98946d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:53 UTC) from {username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:35422
I20260808 07:06:53.249085 28455 tablet_service.cc:1514] Processing DeleteTablet for tablet bbecaf007ec34c9296b7e56fec98946d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:53 UTC) from {username='oryx', principal='oryx/127.18.63.124@KRBTEST.COM'} at 127.0.0.1:59792
I20260808 07:06:53.249266 28706 tablet_replica.cc:330] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: stopping tablet replica
I20260808 07:06:53.249320 28706 raft_consensus.cc:2229] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:53.249539 28706 raft_consensus.cc:2258] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:53.249630 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27911
I20260808 07:06:53.249648 28703 tablet_replica.cc:330] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: stopping tablet replica
I20260808 07:06:53.249725 28703 raft_consensus.cc:2229] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:53.249775 28703 raft_consensus.cc:2258] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:53.251966 28703 ts_tablet_manager.cc:1905] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:53.253703 28703 ts_tablet_manager.cc:1918] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:53.253760 28703 log.cc:1196] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/bbecaf007ec34c9296b7e56fec98946d
I20260808 07:06:53.253968 28703 ts_tablet_manager.cc:1939] T bbecaf007ec34c9296b7e56fec98946d P aff17fccf9fb401b8390ddf41368827c: Deleting consensus metadata
I20260808 07:06:53.254482 27946 catalog_manager.cc:4841] TS aff17fccf9fb401b8390ddf41368827c (127.18.63.66:40055): tablet bbecaf007ec34c9296b7e56fec98946d (table test-table [id=9284d01a59bb4f59b342327441dc0860]) successfully deleted
I20260808 07:06:53.263924 28706 ts_tablet_manager.cc:1905] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:53.265723 28706 ts_tablet_manager.cc:1918] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:53.265797 28706 log.cc:1196] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/bbecaf007ec34c9296b7e56fec98946d
I20260808 07:06:53.266037 28706 ts_tablet_manager.cc:1939] T bbecaf007ec34c9296b7e56fec98946d P e6d1ce42008c48e3bb5dddf51b495626: Deleting consensus metadata
I20260808 07:06:53.266412 27948 catalog_manager.cc:4841] TS e6d1ce42008c48e3bb5dddf51b495626 (127.18.63.67:37761): tablet bbecaf007ec34c9296b7e56fec98946d (table test-table [id=9284d01a59bb4f59b342327441dc0860]) successfully deleted
I20260808 07:06:53.269229 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:53.269079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46189 (local address 127.0.0.1:35436)
0808 07:06:53.269152 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:53.269187 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:06:53.269404 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 28191
W20260808 07:06:53.269423 27947 catalog_manager.cc:4584] TS 42095ed2821f4ddd9b3281ff509327a1 (127.18.63.65:46189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbecaf007ec34c9296b7e56fec98946d: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46189: connect: Connection refused (error 111)
I20260808 07:06:53.280450 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 28373
I20260808 07:06:53.291457 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27557
I20260808 07:06:53.302380 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 27793
2026-08-08T07:06:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6716 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:53 dist-test-slave-bw88 krb5kdc[28712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:53 dist-test-slave-bw88 krb5kdc[28712](info): set up 2 sockets
Aug 08 07:06:53 dist-test-slave-bw88 krb5kdc[28712](info): commencing operation
krb5kdc: starting...
W20260808 07:06:54.491911 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:54 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab.
Entry for principal oryx/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab.
I20260808 07:06:54.615824 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33915
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab
--principal=oryx/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:54.699889 28729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:54.700116 28729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:54.700139 28729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:54.701455 28729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:54.701499 28729 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 07:06:54.701514 28729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:54.701555 28729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:54.701588 28729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:54.701606 28729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:54.703315 28729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33915
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.126.keytab
--principal=oryx/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:54.703584 28729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:54.703831 28729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:54.708938 28735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:54.708925 28736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:54.708923 28739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:54.709184 28729 server_base.cc:1034] running on GCE node
I20260808 07:06:54.709470 28729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:54.709782 28729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:54.710956 28729 hybrid_clock.cc:648] HybridClock initialized: now 1786172814710940 us; error 45 us; skew 500 ppm
Aug 08 07:06:54 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:54.713587 28729 init.cc:377] Logged in from keytab as oryx/127.18.63.126@KRBTEST.COM (short username oryx)
I20260808 07:06:54.714094 28729 webserver.cc:460] Webserver started at http://127.18.63.126:36533/ using document root <none> and password file <none>
I20260808 07:06:54.714354 28729 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:54.714424 28729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:54.714584 28729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:54.715690 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "c264f7980bba4152811356e30ca26406"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "66c01ec8cb10c2ec2bc6ab88bd32092f"
server_key_iv: "8c41e51d6b51c9ae1a2cbb78b6078a0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.716059 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "c264f7980bba4152811356e30ca26406"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "66c01ec8cb10c2ec2bc6ab88bd32092f"
server_key_iv: "8c41e51d6b51c9ae1a2cbb78b6078a0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.717481 28729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:54.718273 28749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:54.718475 28729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:54.718585 28729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "c264f7980bba4152811356e30ca26406"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "66c01ec8cb10c2ec2bc6ab88bd32092f"
server_key_iv: "8c41e51d6b51c9ae1a2cbb78b6078a0b"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.718672 28729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:54.738420 28729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:54.740485 28729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:54.740660 28729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:54.745755 28729 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33915
I20260808 07:06:54.745805 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33915 every 8 connection(s)
I20260808 07:06:54.746140 28729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:54.746800 28831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:54.749145 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Bootstrap starting.
I20260808 07:06:54.749830 28831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:54.750146 28831 log.cc:824] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:54.751034 28831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: No bootstrap required, opened a new log
I20260808 07:06:54.751294 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 28729
I20260808 07:06:54.751420 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:54.751641 18685 external_mini_cluster.cc:1428] Setting key 4cea34e2e13ae8c601ec81a297182305
I20260808 07:06:54.753482 28831 raft_consensus.cc:348] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } }
I20260808 07:06:54.753651 28831 raft_consensus.cc:374] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:54.753700 28831 raft_consensus.cc:729] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c264f7980bba4152811356e30ca26406, State: Initialized, Role: FOLLOWER
I20260808 07:06:54.753855 28831 consensus_queue.cc:260] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } }
I20260808 07:06:54.753947 28831 raft_consensus.cc:388] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:54.753993 28831 raft_consensus.cc:482] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:54.754045 28831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:54.754642 28831 raft_consensus.cc:504] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } }
I20260808 07:06:54.754765 28831 leader_election.cc:302] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c264f7980bba4152811356e30ca26406; no voters:
I20260808 07:06:54.754966 28831 leader_election.cc:288] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:54.755002 28838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:54.755126 28838 raft_consensus.cc:686] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [term 1 LEADER]: Becoming Leader. State: Replica: c264f7980bba4152811356e30ca26406, State: Running, Role: LEADER
I20260808 07:06:54.755211 28838 consensus_queue.cc:237] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } }
I20260808 07:06:54.755246 28831 sys_catalog.cc:564] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:54.755580 28838 sys_catalog.cc:455] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c264f7980bba4152811356e30ca26406. Latest consensus state: current_term: 1 leader_uuid: "c264f7980bba4152811356e30ca26406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } } }
I20260808 07:06:54.755653 28838 sys_catalog.cc:458] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:54.755815 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c264f7980bba4152811356e30ca26406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c264f7980bba4152811356e30ca26406" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33915 } } }
I20260808 07:06:54.755890 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:54.756018 28844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:54.756580 28844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:54.758395 28844 catalog_manager.cc:1347] Generated new cluster ID: 1b71611ddfb94ec9be78f983acedf2fe
I20260808 07:06:54.758451 28844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:06:54 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:54.763711 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:54.752213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.18.63.126:33915)
0808 07:06:54.752393 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:54.752403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:54.752438 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:06:54.753477 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:54.753486 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:54.753541 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:54.753705 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:54.754523 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.755869 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.757016 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.757585 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.757708 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:54.760266 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:54.760270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:54.760270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:54.760282 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:54.762063 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.762437 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.762440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.762443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.762549 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.762796 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.762797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.762798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.763035 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:54.763197 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:54.763306 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 07:06:54.763559 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260808 07:06:54.781522 28844 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:54.782286 28844 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:54.795634 28844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Generated new TSK 0
I20260808 07:06:54.795876 28844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
I20260808 07:06:54.828195 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33915
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:54.913215 28866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:54.913439 28866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:54.913466 28866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:54.914633 28866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:54.914674 28866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:54.914695 28866 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:54.914753 28866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:54.916287 28866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:54.916556 28866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:54.916797 28866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:54.922122 28874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:54.922094 28873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:54.922102 28876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:54.922390 28866 server_base.cc:1034] running on GCE node
I20260808 07:06:54.922660 28866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:54.922873 28866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:54.924018 28866 hybrid_clock.cc:648] HybridClock initialized: now 1786172814924008 us; error 35 us; skew 500 ppm
Aug 08 07:06:54 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:54.926915 28866 init.cc:377] Logged in from keytab as oryx/127.18.63.65@KRBTEST.COM (short username oryx)
I20260808 07:06:54.927420 28866 webserver.cc:460] Webserver started at http://127.18.63.65:42149/ using document root <none> and password file <none>
I20260808 07:06:54.927642 28866 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:54.927688 28866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:54.927786 28866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:54.928907 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "521942989f2fa29e6ea2e1dac1590aaf"
server_key_iv: "11420e96c20dd5e9cf7c8711c00ebcb1"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.929271 28866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "521942989f2fa29e6ea2e1dac1590aaf"
server_key_iv: "11420e96c20dd5e9cf7c8711c00ebcb1"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.930806 28866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:54.931602 28888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:54.931799 28866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:54.931881 28866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
format_stamp: "Formatted at 2026-08-08 07:06:54 on dist-test-slave-bw88"
server_key: "521942989f2fa29e6ea2e1dac1590aaf"
server_key_iv: "11420e96c20dd5e9cf7c8711c00ebcb1"
server_key_version: "encryptionkey@0"
I20260808 07:06:54.931941 28866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:54.956050 28866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:54.958702 28866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:54.958828 28866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:54.959451 28866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:54.959519 28866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:54.959556 28866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:54.959585 28866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:54 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:54.967862 28866 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:34333
I20260808 07:06:54.967952 29036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:34333 every 8 connection(s)
I20260808 07:06:54.968286 28866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:54.969726 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:54.959884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:60063 (local address 127.18.63.126:33915)
0808 07:06:54.959958 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:54.959961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:54.960222 (+ 261us) server_negotiation.cc:407] Connection header received
0808 07:06:54.960723 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:54.960725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:54.960739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:54.960823 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:54.961698 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.962249 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.963301 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.963532 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.963616 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:54.966439 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:54.966441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:54.966441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:54.966454 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:54.968295 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.968811 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.968812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.968813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.968879 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.969142 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.969143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.969143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.969255 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:54.969312 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:54.969579 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:06:54.969660 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:54.969879 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:54.959888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.65:60063)
0808 07:06:54.960068 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:54.960109 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:54.960618 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:54.960926 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:54.960931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:54.961203 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:54.961583 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:54.961589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.962356 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.962357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:54.963197 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:54.963200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.963625 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.963627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:54.963808 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:54.964349 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:54.964358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:54.966313 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:54.968385 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:54.968391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:54.968393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:54.968721 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:54.968981 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:54.968982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:54.968982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:54.969090 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:54.969301 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:54.969304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:54.969465 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 07:06:54.969550 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:06:54.969673 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260808 07:06:54.970431 29037 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33915
I20260808 07:06:54.970525 29037 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:54.970714 29037 heartbeater.cc:511] Master 127.18.63.126:33915 requested a full tablet report, sending...
I20260808 07:06:54.971225 28774 ts_manager.cc:194] Registered new tserver with Master: a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333)
I20260808 07:06:54.971971 28774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:60063
I20260808 07:06:54.974040 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 28866
I20260808 07:06:54.974161 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:54.974397 18685 external_mini_cluster.cc:1428] Setting key 783368b2b50588b44488cbf0eb732085
I20260808 07:06:54.978614 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:54.971687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:34191 (local address 127.18.63.126:33915)
0808 07:06:54.971727 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:54.971729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:54.971880 (+ 151us) server_negotiation.cc:407] Connection header received
0808 07:06:54.971908 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:54.971909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:54.971920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:54.971994 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:54.972454 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.973043 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.974233 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.974566 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.974632 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:54.975580 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:54.975582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:54.975582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:54.975590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:54.977272 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.977744 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.977746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.977746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.977805 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:54.978018 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:54.978019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:54.978020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:54.978149 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:54.978192 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:54.978406 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 07:06:54.978511 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:54.978943 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:54.971635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.65:34191)
0808 07:06:54.971784 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:54.971796 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:54.971858 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:54.972043 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:54.972044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:54.972242 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:54.972316 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:54.972319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.973159 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.973160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:54.974094 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:54.974098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:54.974643 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:54.974644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:54.974712 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:54.975160 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:54.975164 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:54.975446 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:54.977408 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:54.977409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:54.977409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:54.977651 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:54.977884 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:54.977884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:54.977885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:54.977946 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:54.978245 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:54.978246 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:54.978293 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:06:54.978702 (+ 409us) client_negotiation.cc:240] Negotiation successful
0808 07:06:54.978829 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
I20260808 07:06:55.004860 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33915
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:55.092908 29047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:55.093214 29047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:55.093256 29047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:55.094849 29047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:55.094924 29047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:55.094972 29047 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:55.095008 29047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:55.096580 29047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:55.096866 29047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:55.097113 29047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:55.102754 29054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:55.102767 29055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:55.102833 29047 server_base.cc:1034] running on GCE node
W20260808 07:06:55.102782 29058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:55.103340 29047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:55.103601 29047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:55.104769 29047 hybrid_clock.cc:648] HybridClock initialized: now 1786172815104772 us; error 47 us; skew 500 ppm
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:55.107363 29047 init.cc:377] Logged in from keytab as oryx/127.18.63.66@KRBTEST.COM (short username oryx)
I20260808 07:06:55.107810 29047 webserver.cc:460] Webserver started at http://127.18.63.66:39931/ using document root <none> and password file <none>
I20260808 07:06:55.108068 29047 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:55.108139 29047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:55.108282 29047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:55.109397 29047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "0b983cba633a49b2b1086fe00c6e081d"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "23dbe3b2cc93f7103505a57611e3d860"
server_key_iv: "c0b927f7ff313fed054b84586c8028d4"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.109776 29047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "0b983cba633a49b2b1086fe00c6e081d"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "23dbe3b2cc93f7103505a57611e3d860"
server_key_iv: "c0b927f7ff313fed054b84586c8028d4"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.111168 29047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:55.111955 29066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.112128 29047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:55.112231 29047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "0b983cba633a49b2b1086fe00c6e081d"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "23dbe3b2cc93f7103505a57611e3d860"
server_key_iv: "c0b927f7ff313fed054b84586c8028d4"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.112303 29047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:55.129869 29047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:55.131892 29047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:55.132061 29047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:55.132616 29047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:55.132692 29047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.132735 29047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:55.132767 29047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:55.140314 29047 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:41031
I20260808 07:06:55.140370 29218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:41031 every 8 connection(s)
I20260808 07:06:55.140725 29047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:55.143174 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.133132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:52983 (local address 127.18.63.126:33915)
0808 07:06:55.133194 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.133196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.133488 (+ 292us) server_negotiation.cc:407] Connection header received
0808 07:06:55.134024 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.134026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.134042 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.134119 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.134988 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.135859 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.137523 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.137866 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.137959 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.140488 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.140489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.140490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.140497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.142032 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.142401 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.142402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.142403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.142458 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.142633 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.142633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.142633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.142736 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.142788 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.143027 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.143097 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:55.143350 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.133122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.66:52983)
0808 07:06:55.133306 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.133381 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.133917 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.134240 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.134244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.134570 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:55.134879 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.134887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.135951 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.135952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.137425 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.137429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.137984 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.137985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.138138 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.138687 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:55.138696 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:55.140375 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:55.142108 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.142114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.142115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.142333 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.142503 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.142503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.142503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.142584 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.142795 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:55.142801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:55.142915 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.142997 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.143114 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260808 07:06:55.143848 29219 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33915
I20260808 07:06:55.143931 29219 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:55.144117 29219 heartbeater.cc:511] Master 127.18.63.126:33915 requested a full tablet report, sending...
I20260808 07:06:55.144529 28773 ts_manager.cc:194] Registered new tserver with Master: 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031)
I20260808 07:06:55.145072 28773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:52983
I20260808 07:06:55.150259 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.144996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:38257 (local address 127.18.63.126:33915)
0808 07:06:55.145040 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.145042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.145169 (+ 127us) server_negotiation.cc:407] Connection header received
0808 07:06:55.145243 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.145244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.145255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.145316 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.145712 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.146167 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.146877 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.147085 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.147162 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.147956 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.147958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.147958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.147965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.149044 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.149423 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.149424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.149424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.149479 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.149695 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.149696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.149697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.149843 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.149920 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.150093 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.150164 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:55.150274 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.144903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.66:38257)
0808 07:06:55.145086 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.145098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.145178 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.145326 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.145327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.145554 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:55.145617 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.145619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.146278 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.146279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.146792 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.146793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.147170 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.147171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.147239 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.147593 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:55.147597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:55.147866 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:55.149133 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.149134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.149135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.149315 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.149557 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.149558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.149558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.149620 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.149925 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:55.149926 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:55.149973 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.150053 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.150171 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260808 07:06:55.150794 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 29047
I20260808 07:06:55.150911 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:55.151135 18685 external_mini_cluster.cc:1428] Setting key 09f1c998e6b9dd3a1f2f8f5c3bc9f24a
WARNING: no policy specified for oryx/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
I20260808 07:06:55.178043 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33915
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:32893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:55.264153 29231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:55.264384 29231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:55.264427 29231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:55.265667 29231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:55.265726 29231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:55.265764 29231 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 07:06:55.265820 29231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:55.267318 29231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:32893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.63.126:33915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:55.267591 29231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:55.267817 29231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:55.273048 29238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:55.273051 29237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:55.273118 29231 server_base.cc:1034] running on GCE node
W20260808 07:06:55.273084 29240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:55.273592 29231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:55.273797 29231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:55.274932 29231 hybrid_clock.cc:648] HybridClock initialized: now 1786172815274919 us; error 38 us; skew 500 ppm
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:55.277552 29231 init.cc:377] Logged in from keytab as oryx/127.18.63.67@KRBTEST.COM (short username oryx)
I20260808 07:06:55.278015 29231 webserver.cc:460] Webserver started at http://127.18.63.67:44825/ using document root <none> and password file <none>
I20260808 07:06:55.278218 29231 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:55.278258 29231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:55.278348 29231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:55.279316 29231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "634d02277c704f7c8c5c390596ef36a8"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "96615209a8b353237efbdcd05d8e5a12"
server_key_iv: "2dacfbc310e97d39691e98a56063822c"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.279632 29231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "634d02277c704f7c8c5c390596ef36a8"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "96615209a8b353237efbdcd05d8e5a12"
server_key_iv: "2dacfbc310e97d39691e98a56063822c"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.280884 29231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 07:06:55.281625 29248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.281790 29231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:55.281899 29231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "634d02277c704f7c8c5c390596ef36a8"
format_stamp: "Formatted at 2026-08-08 07:06:55 on dist-test-slave-bw88"
server_key: "96615209a8b353237efbdcd05d8e5a12"
server_key_iv: "2dacfbc310e97d39691e98a56063822c"
server_key_version: "encryptionkey@0"
I20260808 07:06:55.281975 29231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:55.292079 29231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:55.294447 29231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:55.294611 29231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:55.295112 29231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:55.295177 29231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.295241 29231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:55.295274 29231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.303437 29231 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:39159
I20260808 07:06:55.303539 29398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:39159 every 8 connection(s)
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:55.303874 29231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:06:55.306844 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.295757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58013 (local address 127.18.63.126:33915)
0808 07:06:55.295813 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.295816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.297520 (+ 1704us) server_negotiation.cc:407] Connection header received
0808 07:06:55.298083 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.298085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.298099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.298157 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.298865 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.299722 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.301383 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.301708 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.301791 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.304350 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.304351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.304352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.304359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.305679 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.306048 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.306049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.306049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.306100 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.306318 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.306318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.306319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.306420 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.306471 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.306716 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.306779 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:06:55.307022 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.295722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.67:58013)
0808 07:06:55.297372 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.297414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.297972 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.298308 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.298313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.298570 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:55.298754 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.298758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.299828 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.299829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.301258 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.301261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.301838 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.301840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.301985 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.302554 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:55.302562 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:55.304223 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:55.305755 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.305762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.305763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.305977 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.306165 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.306166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.306166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.306263 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.306475 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:55.306479 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:55.306588 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.306683 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.306804 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1585,"thread_start_us":1500,"threads_started":1}
I20260808 07:06:55.307514 29399 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33915
I20260808 07:06:55.307600 29399 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:55.307801 29399 heartbeater.cc:511] Master 127.18.63.126:33915 requested a full tablet report, sending...
I20260808 07:06:55.308207 28774 ts_manager.cc:194] Registered new tserver with Master: 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:55.308699 28774 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:58013
I20260808 07:06:55.313831 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 29231
I20260808 07:06:55.313882 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.308684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:39609 (local address 127.18.63.126:33915)
0808 07:06:55.308724 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.308726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.308899 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:06:55.308946 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.308946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.308957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.309021 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.309316 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.309873 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.310540 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.310716 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.310764 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.311557 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.311558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.311558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.311564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.312797 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.313114 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.313115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.313115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.313171 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.313378 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.313379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.313379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.313486 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.313541 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.313768 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.313825 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:06:55.313943 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:55.313987 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.308616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.18.63.67:39609)
0808 07:06:55.308824 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.308835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.308894 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.309024 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.309024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.309194 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:55.309244 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.309246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.309966 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.309967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.310470 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.310472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.310780 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.310780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.310830 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.311197 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:55.311201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:55.311478 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:55.312874 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.312874 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.312875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.313051 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.313230 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.313230 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.313231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.313283 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.313573 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:55.313574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:55.313655 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.313748 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.313866 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260808 07:06:55.314180 18685 external_mini_cluster.cc:1428] Setting key bc4b78238299790954d1f6fa77a47038
I20260808 07:06:55.315258 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:06:55.337968 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:55.332406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.18.63.126:33915)
0808 07:06:55.332462 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.332464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.332514 (+ 50us) server_negotiation.cc:407] Connection header received
0808 07:06:55.332604 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.332605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.332615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.332662 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.333213 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.333764 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.334575 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.334768 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.334819 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.337869 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42206: BlockingRecv error: recv got EOF from 127.0.0.1:42206 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:55.345677 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.338799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.18.63.126:33915)
0808 07:06:55.338855 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.338857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.338943 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:06:55.339009 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.339010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.339022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.339088 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.339547 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.339992 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.340678 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.340855 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.340926 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.343058 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.343059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.343060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.343065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.344498 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.344874 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.344875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.344876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.344937 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.345109 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.345110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.345110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.345217 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.345293 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.345436 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.345527 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:06:55.347314 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:55.348819 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:55.356007 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:39159 (local address 127.0.0.1:56752)
0808 07:06:55.351277 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.351317 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.351381 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.351991 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.351995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.351998 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.352080 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.352082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.353470 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.353471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.354805 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.354808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.355712 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.355713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.355782 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.355793 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.355872 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.355929 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260808 07:06:55.356246 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.18.63.67:39159)
0808 07:06:55.351769 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.351774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.351788 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:06:55.351831 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.351833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.351850 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.351984 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.352158 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.353295 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.354893 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.355633 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.355712 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.355782 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.355907 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.355948 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":201,"threads_started":1}
I20260808 07:06:55.357127 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.0.0.1:37766)
0808 07:06:55.352040 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.352051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.352117 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.352339 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.352340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.352342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.352379 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.352380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.353664 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.353665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.354810 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.354812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.356839 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.356840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.356905 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.356909 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.356975 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.357022 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":27,"thread_start_us":81,"threads_started":1}
I20260808 07:06:55.357133 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.18.63.65:34333)
0808 07:06:55.351653 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.351660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.352194 (+ 534us) server_negotiation.cc:407] Connection header received
0808 07:06:55.352242 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.352244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.352259 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.352333 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.352475 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.353572 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.355820 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.356750 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.356824 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.356893 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.356991 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.357028 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260808 07:06:55.357025 29297 tablet_service.cc:1467] Processing CreateTablet for tablet a98027fa34754d6b9647441bfb43f2f6 (DEFAULT_TABLE table=test-table [id=873d5968f33044438fc51050d184d725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.357498 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98027fa34754d6b9647441bfb43f2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.357717 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:41031 (local address 127.0.0.1:33152)
0808 07:06:55.351471 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.351483 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.351541 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.353708 (+ 2167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.353710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.353712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.353759 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.353761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.354760 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.354762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.355884 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.355887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.357463 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.357464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.357530 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.357535 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.357605 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.357654 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":46,"threads_started":1}
I20260808 07:06:55.357760 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.351832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.18.63.66:41031)
0808 07:06:55.353555 (+ 1723us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.353560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.353572 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:06:55.353612 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.353614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.353632 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.353707 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.353836 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.354661 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.355992 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.357308 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.357462 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.357549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.357620 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.357663 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1680,"thread_start_us":110,"threads_started":1}
I20260808 07:06:55.358040 28954 tablet_service.cc:1467] Processing CreateTablet for tablet a98027fa34754d6b9647441bfb43f2f6 (DEFAULT_TABLE table=test-table [id=873d5968f33044438fc51050d184d725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.358323 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98027fa34754d6b9647441bfb43f2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.358419 29135 tablet_service.cc:1467] Processing CreateTablet for tablet a98027fa34754d6b9647441bfb43f2f6 (DEFAULT_TABLE table=test-table [id=873d5968f33044438fc51050d184d725]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.358770 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98027fa34754d6b9647441bfb43f2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.361541 29435 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Bootstrap starting.
I20260808 07:06:55.362303 29435 tablet_bootstrap.cc:654] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.362291 29436 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Bootstrap starting.
I20260808 07:06:55.362582 29437 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Bootstrap starting.
I20260808 07:06:55.362972 29436 tablet_bootstrap.cc:654] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.363126 29435 log.cc:824] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:55.363296 29437 tablet_bootstrap.cc:654] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.363315 29436 log.cc:824] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:55.363667 29437 log.cc:824] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:55.364158 29435 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: No bootstrap required, opened a new log
I20260808 07:06:55.364241 29435 ts_tablet_manager.cc:1397] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:06:55.364329 29436 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: No bootstrap required, opened a new log
I20260808 07:06:55.364399 29436 ts_tablet_manager.cc:1397] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.364562 29437 tablet_bootstrap.cc:492] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: No bootstrap required, opened a new log
I20260808 07:06:55.364647 29437 ts_tablet_manager.cc:1397] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.365990 29436 raft_consensus.cc:348] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.366154 29436 raft_consensus.cc:374] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.366200 29436 raft_consensus.cc:729] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.366312 29436 consensus_queue.cc:260] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.366364 29435 raft_consensus.cc:348] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.366547 29435 raft_consensus.cc:374] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.366595 29435 raft_consensus.cc:729] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.366616 29436 ts_tablet_manager.cc:1428] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:06:55.366760 29037 heartbeater.cc:503] Master 127.18.63.126:33915 was elected leader, sending a full tablet report...
I20260808 07:06:55.366711 29435 consensus_queue.cc:260] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.366757 29437 raft_consensus.cc:348] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.366905 29437 raft_consensus.cc:374] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.366969 29437 raft_consensus.cc:729] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.367086 29437 consensus_queue.cc:260] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.367378 29437 ts_tablet_manager.cc:1428] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 07:06:55.367440 29219 heartbeater.cc:503] Master 127.18.63.126:33915 was elected leader, sending a full tablet report...
I20260808 07:06:55.367697 29435 ts_tablet_manager.cc:1428] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:06:55.367708 29399 heartbeater.cc:503] Master 127.18.63.126:33915 was elected leader, sending a full tablet report...
I20260808 07:06:55.376255 29445 raft_consensus.cc:482] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:55.376343 29445 raft_consensus.cc:504] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.376605 29442 raft_consensus.cc:482] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:55.376721 29442 raft_consensus.cc:504] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.376762 29445 leader_election.cc:288] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333)
I20260808 07:06:55.377065 29442 leader_election.cc:288] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159), 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031)
I20260808 07:06:55.381181 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:41031 (local address 127.18.63.65:34685)
0808 07:06:55.377306 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.377316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.377355 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.377704 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.377704 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.377706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.377765 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.377767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378706 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.378707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.379903 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.379906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380898 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.380995 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.380998 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.381029 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.381071 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:06:55.381251 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.376933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:41031 (local address 127.18.63.67:52301)
0808 07:06:55.376993 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.377005 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.377047 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.377314 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.377315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.377318 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.377416 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.377418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378382 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.378383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.379610 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.379613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380454 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.381027 (+ 572us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.381032 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.381094 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.381153 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:06:55.381435 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:36307 (local address 127.18.63.65:34333)
0808 07:06:55.377335 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.377336 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.377344 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:55.377366 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.377377 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.377386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.377486 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.377847 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378595 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380338 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380853 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380896 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.380904 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.381146 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.381195 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:06:55.381487 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46187 (local address 127.18.63.67:39159)
0808 07:06:55.377794 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.377795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.377802 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:55.377820 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.377821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.377828 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.377871 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.378059 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378740 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380477 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.381074 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.381113 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.381119 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.381397 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.381438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:06:55.381551 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52301 (local address 127.18.63.66:41031)
0808 07:06:55.377070 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.377072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.377083 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:06:55.377113 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.377114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.377124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.377192 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.377507 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378288 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.379770 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380342 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380411 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.380421 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.381442 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.381481 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:55.381712 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:39159 (local address 127.18.63.65:46187)
0808 07:06:55.377601 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.377610 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.377664 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.377906 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.377907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.377919 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.377990 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.377992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378838 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.378839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.380380 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.380384 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.381199 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.381200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.381277 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.381281 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.381553 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.381601 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":197,"thread_start_us":99,"threads_started":1}
I20260808 07:06:55.381799 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.377321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:34685 (local address 127.18.63.66:41031)
0808 07:06:55.377573 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.377575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.377582 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:06:55.377602 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.377603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.377611 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.377831 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:55.377867 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378608 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380024 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380799 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.381611 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.381623 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.381646 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.381693 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":124,"threads_started":1}
I20260808 07:06:55.381821 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d" is_pre_election: true
I20260808 07:06:55.381992 29446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.376945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.18.63.67:36307)
0808 07:06:55.377141 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.377152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.377197 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.377667 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.377668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.377670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:55.377759 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.377761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.378701 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.378702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.380248 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.380251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.380985 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.380986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.381064 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.381067 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.381211 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.381880 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":128,"threads_started":1}
I20260808 07:06:55.382015 29159 raft_consensus.cc:2454] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 0.
I20260808 07:06:55.382114 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d" is_pre_election: true
I20260808 07:06:55.382218 29158 raft_consensus.cc:2454] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.382297 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" is_pre_election: true
I20260808 07:06:55.382325 29249 leader_election.cc:302] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, 634d02277c704f7c8c5c390596ef36a8; no voters:
I20260808 07:06:55.382406 28979 raft_consensus.cc:2454] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 0.
I20260808 07:06:55.382390 28890 leader_election.cc:302] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.382498 29442 raft_consensus.cc:2781] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:55.382556 29442 raft_consensus.cc:482] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:55.382583 29442 raft_consensus.cc:3037] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.382601 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8" is_pre_election: true
I20260808 07:06:55.382656 29445 raft_consensus.cc:2781] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:55.382754 29343 raft_consensus.cc:2454] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.382800 29445 raft_consensus.cc:482] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:55.382840 29445 raft_consensus.cc:3037] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.383549 29442 raft_consensus.cc:504] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.383723 29442 leader_election.cc:288] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Requested vote from peers 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159), 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031)
I20260808 07:06:55.383694 29445 raft_consensus.cc:504] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.383814 29445 leader_election.cc:288] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 election: Requested vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333)
I20260808 07:06:55.384020 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
I20260808 07:06:55.384034 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d"
I20260808 07:06:55.384019 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8"
I20260808 07:06:55.384094 29159 raft_consensus.cc:3037] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.384111 28979 raft_consensus.cc:2379] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 634d02277c704f7c8c5c390596ef36a8 in current term 1: Already voted for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in this term.
I20260808 07:06:55.384126 29343 raft_consensus.cc:2379] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in current term 1: Already voted for candidate 634d02277c704f7c8c5c390596ef36a8 in this term.
I20260808 07:06:55.384228 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98027fa34754d6b9647441bfb43f2f6" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d"
I20260808 07:06:55.384521 28890 leader_election.cc:302] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters: 0b983cba633a49b2b1086fe00c6e081d, 634d02277c704f7c8c5c390596ef36a8
I20260808 07:06:55.384631 29442 raft_consensus.cc:2726] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 07:06:55.384985 29159 raft_consensus.cc:2454] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 1.
I20260808 07:06:55.385214 29249 leader_election.cc:302] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, 634d02277c704f7c8c5c390596ef36a8; no voters: a6fee4c8cbc34f2a92c39fd6b1d978d0
I20260808 07:06:55.385308 29445 raft_consensus.cc:2781] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:55.385501 29445 raft_consensus.cc:686] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 LEADER]: Becoming Leader. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Running, Role: LEADER
I20260808 07:06:55.385610 29445 consensus_queue.cc:237] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } }
I20260808 07:06:55.386412 28773 catalog_manager.cc:5495] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "634d02277c704f7c8c5c390596ef36a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } health_report { overall_health: UNKNOWN } } }
W20260808 07:06:55.391662 29221 tablet.cc:2369] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:55.391867 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.389449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.18.63.67:39159)
0808 07:06:55.389496 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.389498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.389507 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:55.389576 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.389576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.389593 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.389691 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:55.389833 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.390287 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.391050 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.391256 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.391328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.391517 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.391643 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.391736 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.391776 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.391794 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:06:55.393505 29159 raft_consensus.cc:1261] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Refusing update from remote peer 634d02277c704f7c8c5c390596ef36a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.393549 28979 raft_consensus.cc:1261] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Refusing update from remote peer 634d02277c704f7c8c5c390596ef36a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.393756 29445 consensus_queue.cc:1035] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.393918 29450 consensus_queue.cc:1035] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.396188 29455 mvcc.cc:205] Tried to move back new op lower bound from 7316163851849469952 to 7316163851820118016. Current Snapshot: MvccSnapshot[applied={T|T < 7316163851849469952}]
I20260808 07:06:55.396458 29459 mvcc.cc:205] Tried to move back new op lower bound from 7316163851849469952 to 7316163851820118016. Current Snapshot: MvccSnapshot[applied={T|T < 7316163851849469952}]
I20260808 07:06:55.396883 29458 mvcc.cc:205] Tried to move back new op lower bound from 7316163851849469952 to 7316163851820118016. Current Snapshot: MvccSnapshot[applied={T|T < 7316163851849469952}]
I20260808 07:06:55.398546 28773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.398650 28773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.399406 28773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Sending DeleteTablet for 3 replicas of tablet a98027fa34754d6b9647441bfb43f2f6
I20260808 07:06:55.399782 29297 tablet_service.cc:1514] Processing DeleteTablet for tablet a98027fa34754d6b9647441bfb43f2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 07:06:55.399811 29135 tablet_service.cc:1514] Processing DeleteTablet for tablet a98027fa34754d6b9647441bfb43f2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260808 07:06:55.400031 28954 tablet_service.cc:1514] Processing DeleteTablet for tablet a98027fa34754d6b9647441bfb43f2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37766
I20260808 07:06:55.400055 29469 tablet_replica.cc:330] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: stopping tablet replica
I20260808 07:06:55.400190 29469 raft_consensus.cc:2229] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:55.400115 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:55.400264 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:55.400378 29469 raft_consensus.cc:2258] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.400569 29468 tablet_replica.cc:330] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: stopping tablet replica
I20260808 07:06:55.400699 29468 raft_consensus.cc:2229] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.400830 29472 tablet_replica.cc:330] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: stopping tablet replica
I20260808 07:06:55.400861 29468 raft_consensus.cc:2258] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.400936 29472 raft_consensus.cc:2229] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.400949 29469 ts_tablet_manager.cc:1905] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.401058 29472 raft_consensus.cc:2258] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.401393 29468 ts_tablet_manager.cc:1905] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.401541 28954 tablet_service.cc:1467] Processing CreateTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d (DEFAULT_TABLE table=test-table [id=9cb25413da654d2aadf0ad8405d6d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.401671 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ba4d6e9f5247dab39f8042bc70305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.401697 29135 tablet_service.cc:1467] Processing CreateTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d (DEFAULT_TABLE table=test-table [id=9cb25413da654d2aadf0ad8405d6d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.401813 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ba4d6e9f5247dab39f8042bc70305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.402668 29436 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Bootstrap starting.
I20260808 07:06:55.402897 29297 tablet_service.cc:1467] Processing CreateTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d (DEFAULT_TABLE table=test-table [id=9cb25413da654d2aadf0ad8405d6d498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.403101 29436 tablet_bootstrap.cc:654] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.403228 29437 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Bootstrap starting.
I20260808 07:06:55.403708 29437 tablet_bootstrap.cc:654] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.403863 29472 ts_tablet_manager.cc:1905] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.405150 29472 ts_tablet_manager.cc:1918] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.405189 29468 ts_tablet_manager.cc:1918] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.405249 29472 log.cc:1196] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/a98027fa34754d6b9647441bfb43f2f6
I20260808 07:06:55.405256 29468 log.cc:1196] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/a98027fa34754d6b9647441bfb43f2f6
I20260808 07:06:55.405519 29472 ts_tablet_manager.cc:1939] T a98027fa34754d6b9647441bfb43f2f6 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting consensus metadata
I20260808 07:06:55.405582 29468 ts_tablet_manager.cc:1939] T a98027fa34754d6b9647441bfb43f2f6 P 0b983cba633a49b2b1086fe00c6e081d: Deleting consensus metadata
I20260808 07:06:55.405952 28751 catalog_manager.cc:4841] TS a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333): tablet a98027fa34754d6b9647441bfb43f2f6 (table test-table [id=873d5968f33044438fc51050d184d725]) successfully deleted
I20260808 07:06:55.405990 28750 catalog_manager.cc:4841] TS 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031): tablet a98027fa34754d6b9647441bfb43f2f6 (table test-table [id=873d5968f33044438fc51050d184d725]) successfully deleted
I20260808 07:06:55.406497 29437 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: No bootstrap required, opened a new log
I20260808 07:06:55.406549 29437 ts_tablet_manager.cc:1397] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 07:06:55.406663 29436 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: No bootstrap required, opened a new log
I20260808 07:06:55.406708 29436 ts_tablet_manager.cc:1397] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:06:55.406693 29437 raft_consensus.cc:348] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.406764 29437 raft_consensus.cc:374] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.406785 29437 raft_consensus.cc:729] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.406855 29437 consensus_queue.cc:260] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.406843 29436 raft_consensus.cc:348] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.406909 29436 raft_consensus.cc:374] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.406932 29436 raft_consensus.cc:729] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.406945 29437 ts_tablet_manager.cc:1428] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.406976 29436 consensus_queue.cc:260] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.407061 29436 ts_tablet_manager.cc:1428] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:55.407372 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ba4d6e9f5247dab39f8042bc70305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.408437 29435 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Bootstrap starting.
I20260808 07:06:55.408485 29469 ts_tablet_manager.cc:1918] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.408552 29469 log.cc:1196] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/a98027fa34754d6b9647441bfb43f2f6
I20260808 07:06:55.408835 29469 ts_tablet_manager.cc:1939] T a98027fa34754d6b9647441bfb43f2f6 P 634d02277c704f7c8c5c390596ef36a8: Deleting consensus metadata
I20260808 07:06:55.408910 29435 tablet_bootstrap.cc:654] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.409268 28751 catalog_manager.cc:4841] TS 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159): tablet a98027fa34754d6b9647441bfb43f2f6 (table test-table [id=873d5968f33044438fc51050d184d725]) successfully deleted
I20260808 07:06:55.409703 29435 tablet_bootstrap.cc:492] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: No bootstrap required, opened a new log
I20260808 07:06:55.409765 29435 ts_tablet_manager.cc:1397] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:55.409893 29435 raft_consensus.cc:348] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.409966 29435 raft_consensus.cc:374] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.409991 29435 raft_consensus.cc:729] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.410043 29435 consensus_queue.cc:260] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.410136 29435 ts_tablet_manager.cc:1428] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:06:55.453505 29450 raft_consensus.cc:482] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:55.453647 29450 raft_consensus.cc:504] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.453856 29450 leader_election.cc:288] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333), 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031)
I20260808 07:06:55.454048 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ba4d6e9f5247dab39f8042bc70305d" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" is_pre_election: true
I20260808 07:06:55.454172 28979 raft_consensus.cc:2454] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 0.
I20260808 07:06:55.454087 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ba4d6e9f5247dab39f8042bc70305d" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d" is_pre_election: true
I20260808 07:06:55.454234 29159 raft_consensus.cc:2454] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 0.
I20260808 07:06:55.454334 29250 leader_election.cc:302] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634d02277c704f7c8c5c390596ef36a8, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.454432 29450 raft_consensus.cc:2781] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:55.454483 29450 raft_consensus.cc:482] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:55.454514 29450 raft_consensus.cc:3037] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.455336 29450 raft_consensus.cc:504] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.455519 29450 leader_election.cc:288] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 election: Requested vote from peers a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333), 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031)
I20260808 07:06:55.455659 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ba4d6e9f5247dab39f8042bc70305d" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
I20260808 07:06:55.455689 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ba4d6e9f5247dab39f8042bc70305d" candidate_uuid: "634d02277c704f7c8c5c390596ef36a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d"
I20260808 07:06:55.455753 28979 raft_consensus.cc:3037] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.455766 29159 raft_consensus.cc:3037] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.456502 29159 raft_consensus.cc:2454] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 1.
I20260808 07:06:55.456676 29249 leader_election.cc:302] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, 634d02277c704f7c8c5c390596ef36a8; no voters:
I20260808 07:06:55.456684 28979 raft_consensus.cc:2454] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634d02277c704f7c8c5c390596ef36a8 in term 1.
I20260808 07:06:55.456784 29450 raft_consensus.cc:2781] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:55.456833 29450 raft_consensus.cc:686] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 1 LEADER]: Becoming Leader. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Running, Role: LEADER
I20260808 07:06:55.456889 29450 consensus_queue.cc:237] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.457495 28773 catalog_manager.cc:5495] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "634d02277c704f7c8c5c390596ef36a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } health_report { overall_health: HEALTHY } } }
W20260808 07:06:55.469060 29038 tablet.cc:2369] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172814, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.126@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:55.486126 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.478167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.18.63.126:33915)
0808 07:06:55.478240 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.478243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.478413 (+ 170us) server_negotiation.cc:407] Connection header received
0808 07:06:55.478494 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.478495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.478504 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.478556 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.479215 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.479716 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.480786 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.480963 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.481012 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.483333 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.483335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.483335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.483342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.484804 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.485217 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.485218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.485219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.485274 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.485519 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.485521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.485521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.485636 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.485717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.485872 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.486044 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":44}
I20260808 07:06:55.486120 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.478102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33915 (local address 127.0.0.1:42218)
0808 07:06:55.478309 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.478321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.478422 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.478586 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.478587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.479023 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:55.479083 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.479086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.479815 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.479817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.480694 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.480698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.481053 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.481054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.481119 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.481679 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:55.481684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:55.483243 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:55.484905 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.484912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.484914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.485131 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.485390 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:55.485390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:55.485391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:55.485455 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:55.485706 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:55.485713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:55.485759 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.485847 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.485978 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260808 07:06:55.488415 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:42218:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 07:06:55.488623 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:55.490269 28954 tablet_service.cc:1467] Processing CreateTablet for tablet 573ae6710c1f4643b54383b5201a9119 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27c4680c4af54e08b4dfc13bf0b025b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:55.490305 29297 tablet_service.cc:1467] Processing CreateTablet for tablet 573ae6710c1f4643b54383b5201a9119 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27c4680c4af54e08b4dfc13bf0b025b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:55.490435 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 573ae6710c1f4643b54383b5201a9119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.490437 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 573ae6710c1f4643b54383b5201a9119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.490382 29135 tablet_service.cc:1467] Processing CreateTablet for tablet 573ae6710c1f4643b54383b5201a9119 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=27c4680c4af54e08b4dfc13bf0b025b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 07:06:55.490558 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 573ae6710c1f4643b54383b5201a9119. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.491907 29436 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Bootstrap starting.
I20260808 07:06:55.491991 29435 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: Bootstrap starting.
I20260808 07:06:55.492393 29436 tablet_bootstrap.cc:654] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.492689 29435 tablet_bootstrap.cc:654] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.492796 29437 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: Bootstrap starting.
I20260808 07:06:55.493274 29437 tablet_bootstrap.cc:654] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.493652 29435 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: No bootstrap required, opened a new log
I20260808 07:06:55.493705 29435 ts_tablet_manager.cc:1397] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.493865 29435 raft_consensus.cc:348] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.493932 29435 raft_consensus.cc:374] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.493952 29435 raft_consensus.cc:729] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.493993 29435 consensus_queue.cc:260] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.494091 29435 ts_tablet_manager.cc:1428] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.494138 29436 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: No bootstrap required, opened a new log
I20260808 07:06:55.494181 29436 ts_tablet_manager.cc:1397] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.494135 29450 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.494232 29450 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.494335 29436 raft_consensus.cc:348] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.494418 29436 raft_consensus.cc:374] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.494437 29436 raft_consensus.cc:729] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.494469 29436 consensus_queue.cc:260] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.494551 29436 ts_tablet_manager.cc:1428] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.494588 29442 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.494683 29442 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.494918 29437 tablet_bootstrap.cc:492] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: No bootstrap required, opened a new log
I20260808 07:06:55.494967 29437 ts_tablet_manager.cc:1397] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:55.495345 29437 raft_consensus.cc:348] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.495437 29437 raft_consensus.cc:374] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.495462 29437 raft_consensus.cc:729] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.495503 29437 consensus_queue.cc:260] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.495615 29437 ts_tablet_manager.cc:1428] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:55.495838 29443 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.495924 29443 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 07:06:55.554502 29400 tablet.cc:2369] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:06:55.714277 29442 raft_consensus.cc:482] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:55.714417 29442 raft_consensus.cc:504] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.714663 29442 leader_election.cc:288] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:55.714879 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "573ae6710c1f4643b54383b5201a9119" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d" is_pre_election: true
I20260808 07:06:55.714931 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "573ae6710c1f4643b54383b5201a9119" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8" is_pre_election: true
I20260808 07:06:55.715022 29159 raft_consensus.cc:2454] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.715078 29343 raft_consensus.cc:2454] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.715294 28890 leader_election.cc:302] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.715405 29442 raft_consensus.cc:2781] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:55.715446 29442 raft_consensus.cc:482] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:55.715466 29442 raft_consensus.cc:3037] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.716192 29442 raft_consensus.cc:504] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.716342 29442 leader_election.cc:288] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Requested vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:55.716527 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "573ae6710c1f4643b54383b5201a9119" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d"
I20260808 07:06:55.716552 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "573ae6710c1f4643b54383b5201a9119" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8"
I20260808 07:06:55.716609 29159 raft_consensus.cc:3037] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.716620 29343 raft_consensus.cc:3037] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.717410 29159 raft_consensus.cc:2454] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 1.
I20260808 07:06:55.717526 29343 raft_consensus.cc:2454] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 1.
I20260808 07:06:55.717584 28890 leader_election.cc:302] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.717680 29442 raft_consensus.cc:2781] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:55.717813 29442 raft_consensus.cc:686] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 LEADER]: Becoming Leader. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Running, Role: LEADER
I20260808 07:06:55.717881 29442 consensus_queue.cc:237] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.718243 29488 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: TxnStatusTablet state changed. Reason: New leader a6fee4c8cbc34f2a92c39fd6b1d978d0. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.718348 29488 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:55.718523 29492 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 07:06:55.718690 29492 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 07:06:55.718672 28773 catalog_manager.cc:5495] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:55.780030 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.777177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.18.63.65:34333)
0808 07:06:55.777244 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.777246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.777262 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:06:55.777377 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.777378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.777400 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.777466 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:55.777663 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.778169 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.779238 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.779420 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.779479 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.779687 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.779787 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.779839 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.779925 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.779943 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:06:55.780052 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.777081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.0.0.1:37776)
0808 07:06:55.777178 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.777195 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.777251 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.777498 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.777499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.777501 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:55.777558 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.777561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.778258 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.778260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.779144 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.779148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.779502 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.779503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.779574 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.779592 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.779853 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.779862 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.779921 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.779972 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 07:06:55.781317 29159 raft_consensus.cc:1261] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Refusing update from remote peer a6fee4c8cbc34f2a92c39fd6b1d978d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.781405 29343 raft_consensus.cc:1261] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Refusing update from remote peer a6fee4c8cbc34f2a92c39fd6b1d978d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.781656 29488 consensus_queue.cc:1035] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.781796 29442 consensus_queue.cc:1035] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.782445 29450 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: TxnStatusTablet state changed. Reason: New leader a6fee4c8cbc34f2a92c39fd6b1d978d0. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.782500 29443 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: TxnStatusTablet state changed. Reason: New leader a6fee4c8cbc34f2a92c39fd6b1d978d0. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.782560 29450 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.782595 29443 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.783496 29488 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.783589 29488 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:55.783543 29499 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.783630 29499 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P a6fee4c8cbc34f2a92c39fd6b1d978d0: This TxnStatusTablet replica's current role is: LEADER
I20260808 07:06:55.784140 29443 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.784231 29443 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 0b983cba633a49b2b1086fe00c6e081d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.784343 29450 tablet_replica.cc:439] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } } }
I20260808 07:06:55.784425 29450 tablet_replica.cc:442] T 573ae6710c1f4643b54383b5201a9119 P 634d02277c704f7c8c5c390596ef36a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 07:06:55.789582 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.787222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:48237 (local address 127.18.63.65:34333)
0808 07:06:55.787266 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.787268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.787278 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:55.787335 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.787336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.787346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.787401 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:55.787519 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.787982 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.788802 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.789018 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.789103 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.789270 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.789310 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.789416 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.789492 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.789514 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:55.789613 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.787141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.18.63.67:48237)
0808 07:06:55.787217 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.787228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.787274 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.787404 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.787404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.787405 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:55.787452 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.787454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.788065 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.788066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.788710 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.788712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.789102 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.789104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.789163 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.789187 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.789421 (+ 234us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.789424 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.789486 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.789540 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 07:06:55.791828 29159 raft_consensus.cc:1261] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Refusing update from remote peer 634d02277c704f7c8c5c390596ef36a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.791844 28979 raft_consensus.cc:1261] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Refusing update from remote peer 634d02277c704f7c8c5c390596ef36a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.792039 29445 consensus_queue.cc:1035] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.792158 29450 consensus_queue.cc:1035] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.803844 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.798426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:39159 (local address 127.18.63.65:57307)
0808 07:06:55.798609 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:55.798619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:55.798673 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:55.798819 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:55.798820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:55.798821 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:55.798882 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.798883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.799519 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.799521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.800474 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:55.800477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.801202 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.801203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:55.801295 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.801332 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.803556 (+ 2224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.803559 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:55.803627 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:06:55.803690 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260808 07:06:55.803855 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.798526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:57307 (local address 127.18.63.67:39159)
0808 07:06:55.798603 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.798605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.798657 (+ 52us) server_negotiation.cc:407] Connection header received
0808 07:06:55.798747 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.798747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.798760 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.798815 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:55.798964 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.799433 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.800643 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.801084 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.801193 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.803334 (+ 2141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.803440 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.803553 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.803647 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.803676 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:06:55.812613 28773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.812732 28773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.813508 28773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Sending DeleteTablet for 3 replicas of tablet 27ba4d6e9f5247dab39f8042bc70305d
I20260808 07:06:55.813862 28954 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37766
I20260808 07:06:55.814018 29472 tablet_replica.cc:330] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: stopping tablet replica
I20260808 07:06:55.814533 29472 raft_consensus.cc:2229] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.814586 29472 raft_consensus.cc:2258] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.813923 29135 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260808 07:06:55.814723 29468 tablet_replica.cc:330] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: stopping tablet replica
I20260808 07:06:55.814806 29468 raft_consensus.cc:2229] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.814852 29468 raft_consensus.cc:2258] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.815130 29472 ts_tablet_manager.cc:1905] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.816615 29472 ts_tablet_manager.cc:1918] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:55.816618 29297 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ba4d6e9f5247dab39f8042bc70305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 07:06:55.816682 29472 log.cc:1196] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/27ba4d6e9f5247dab39f8042bc70305d
I20260808 07:06:55.816944 29472 ts_tablet_manager.cc:1939] T 27ba4d6e9f5247dab39f8042bc70305d P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting consensus metadata
I20260808 07:06:55.817281 28751 catalog_manager.cc:4841] TS a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333): tablet 27ba4d6e9f5247dab39f8042bc70305d (table test-table [id=9cb25413da654d2aadf0ad8405d6d498]) successfully deleted
I20260808 07:06:55.817804 29469 tablet_replica.cc:330] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: stopping tablet replica
I20260808 07:06:55.817894 29469 raft_consensus.cc:2229] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:55.817960 29469 raft_consensus.cc:2258] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.818261 29469 ts_tablet_manager.cc:1905] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.818379 29468 ts_tablet_manager.cc:1905] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.820041 29468 ts_tablet_manager.cc:1918] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:55.820094 29468 log.cc:1196] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/27ba4d6e9f5247dab39f8042bc70305d
I20260808 07:06:55.820154 29469 ts_tablet_manager.cc:1918] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:55.820214 29469 log.cc:1196] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/27ba4d6e9f5247dab39f8042bc70305d
I20260808 07:06:55.820309 29468 ts_tablet_manager.cc:1939] T 27ba4d6e9f5247dab39f8042bc70305d P 0b983cba633a49b2b1086fe00c6e081d: Deleting consensus metadata
I20260808 07:06:55.820447 29469 ts_tablet_manager.cc:1939] T 27ba4d6e9f5247dab39f8042bc70305d P 634d02277c704f7c8c5c390596ef36a8: Deleting consensus metadata
I20260808 07:06:55.820823 28750 catalog_manager.cc:4841] TS 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031): tablet 27ba4d6e9f5247dab39f8042bc70305d (table test-table [id=9cb25413da654d2aadf0ad8405d6d498]) successfully deleted
I20260808 07:06:55.821116 28751 catalog_manager.cc:4841] TS 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159): tablet 27ba4d6e9f5247dab39f8042bc70305d (table test-table [id=9cb25413da654d2aadf0ad8405d6d498]) successfully deleted
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:06:55.842891 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:55.837202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.18.63.126:33915)
0808 07:06:55.837256 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.837258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.837407 (+ 149us) server_negotiation.cc:407] Connection header received
0808 07:06:55.837501 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.837502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.837513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.837586 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.838168 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.838661 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.839420 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.839614 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.839716 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.842815 (+ 3099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42228: BlockingRecv error: recv got EOF from 127.0.0.1:42228 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 07:06:55 dist-test-slave-bw88 krb5kdc[28712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM
I20260808 07:06:55.851531 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.843954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.18.63.126:33915)
0808 07:06:55.843996 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.843998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.844008 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:06:55.844059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.844060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.844070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.844146 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:55.844581 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.845373 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.846047 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.846337 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.846408 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.848715 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:55.848716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:55.848717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:55.848725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:55.850432 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.850782 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.850783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.850783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.850850 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:55.851037 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:55.851038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:55.851038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:55.851175 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:55.851275 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.851353 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.851431 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:06:55.853123 28773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:55.853305 28773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:55.854830 28954 tablet_service.cc:1467] Processing CreateTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 (DEFAULT_TABLE table=test-table [id=f0829c4615df4d61add9a0675048874b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.854846 29135 tablet_service.cc:1467] Processing CreateTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 (DEFAULT_TABLE table=test-table [id=f0829c4615df4d61add9a0675048874b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.854846 29297 tablet_service.cc:1467] Processing CreateTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 (DEFAULT_TABLE table=test-table [id=f0829c4615df4d61add9a0675048874b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.854977 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6d57e2650d04ef08ca0012aae01ce66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.854984 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6d57e2650d04ef08ca0012aae01ce66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.854984 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6d57e2650d04ef08ca0012aae01ce66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.856400 29436 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Bootstrap starting.
I20260808 07:06:55.856438 29437 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Bootstrap starting.
I20260808 07:06:55.856560 29435 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Bootstrap starting.
I20260808 07:06:55.856832 29436 tablet_bootstrap.cc:654] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.856850 29437 tablet_bootstrap.cc:654] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.856865 29435 tablet_bootstrap.cc:654] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.857937 29436 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: No bootstrap required, opened a new log
I20260808 07:06:55.857997 29436 ts_tablet_manager.cc:1397] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.858178 29436 raft_consensus.cc:348] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.858301 29436 raft_consensus.cc:374] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.858330 29436 raft_consensus.cc:729] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.858397 29437 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: No bootstrap required, opened a new log
I20260808 07:06:55.858384 29436 consensus_queue.cc:260] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.858436 29437 ts_tablet_manager.cc:1397] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:55.858491 29436 ts_tablet_manager.cc:1428] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.858575 29437 raft_consensus.cc:348] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.858670 29437 raft_consensus.cc:374] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.858711 29437 raft_consensus.cc:729] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.858732 29435 tablet_bootstrap.cc:492] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: No bootstrap required, opened a new log
I20260808 07:06:55.858774 29435 ts_tablet_manager.cc:1397] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:06:55.858753 29437 consensus_queue.cc:260] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.858851 29437 ts_tablet_manager.cc:1428] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:55.858893 29435 raft_consensus.cc:348] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.858973 29435 raft_consensus.cc:374] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.858996 29435 raft_consensus.cc:729] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.859032 29435 consensus_queue.cc:260] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.859120 29435 ts_tablet_manager.cc:1428] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.918665 29499 raft_consensus.cc:482] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:55.918764 29499 raft_consensus.cc:504] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.918984 29499 leader_election.cc:288] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:55.919196 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6d57e2650d04ef08ca0012aae01ce66" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d" is_pre_election: true
I20260808 07:06:55.919281 29159 raft_consensus.cc:2454] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.919206 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6d57e2650d04ef08ca0012aae01ce66" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8" is_pre_election: true
I20260808 07:06:55.919358 29343 raft_consensus.cc:2454] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 0.
I20260808 07:06:55.919443 28890 leader_election.cc:302] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.919559 29499 raft_consensus.cc:2781] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:55.919613 29499 raft_consensus.cc:482] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:55.919646 29499 raft_consensus.cc:3037] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.920351 29499 raft_consensus.cc:504] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.920491 29499 leader_election.cc:288] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Requested vote from peers 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:55.920641 29159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6d57e2650d04ef08ca0012aae01ce66" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b983cba633a49b2b1086fe00c6e081d"
I20260808 07:06:55.920717 29159 raft_consensus.cc:3037] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.920681 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6d57e2650d04ef08ca0012aae01ce66" candidate_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8"
I20260808 07:06:55.920792 29343 raft_consensus.cc:3037] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:55.921416 29159 raft_consensus.cc:2454] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 1.
I20260808 07:06:55.921622 28890 leader_election.cc:302] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:55.921665 29343 raft_consensus.cc:2454] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6fee4c8cbc34f2a92c39fd6b1d978d0 in term 1.
I20260808 07:06:55.921749 29499 raft_consensus.cc:2781] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:55.921799 29499 raft_consensus.cc:686] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 LEADER]: Becoming Leader. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Running, Role: LEADER
I20260808 07:06:55.921845 29499 consensus_queue.cc:237] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.922451 28773 catalog_manager.cc:5495] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:55.933003 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:55.930725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.18.63.65:34333)
0808 07:06:55.930772 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:55.930775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:55.930812 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:06:55.930892 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:55.930893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:55.930906 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:55.930959 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:55.931073 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.931535 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.932277 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:55.932471 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:55.932543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:55.932736 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:55.932775 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:55.932837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:55.932900 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:06:55.932923 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:06:55.933918 29343 raft_consensus.cc:1261] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Refusing update from remote peer a6fee4c8cbc34f2a92c39fd6b1d978d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.934078 29159 raft_consensus.cc:1261] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Refusing update from remote peer a6fee4c8cbc34f2a92c39fd6b1d978d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:55.934121 29442 consensus_queue.cc:1035] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.934271 29442 consensus_queue.cc:1035] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:55.937716 28772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.937794 28772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:55.938457 28772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Sending DeleteTablet for 3 replicas of tablet c6d57e2650d04ef08ca0012aae01ce66
I20260808 07:06:55.938735 28954 tablet_service.cc:1514] Processing DeleteTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37766
I20260808 07:06:55.938728 29135 tablet_service.cc:1514] Processing DeleteTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260808 07:06:55.938815 29472 tablet_replica.cc:330] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: stopping tablet replica
I20260808 07:06:55.938815 29468 tablet_replica.cc:330] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: stopping tablet replica
I20260808 07:06:55.938858 29468 raft_consensus.cc:2229] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.938863 29472 raft_consensus.cc:2229] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:55.938903 29468 raft_consensus.cc:2258] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.938934 29472 raft_consensus.cc:2258] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.938885 28772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:06:55.939036 28772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:06:55.939265 29297 tablet_service.cc:1514] Processing DeleteTablet for tablet c6d57e2650d04ef08ca0012aae01ce66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:55 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 07:06:55.939347 29469 tablet_replica.cc:330] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: stopping tablet replica
I20260808 07:06:55.939365 29468 ts_tablet_manager.cc:1905] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.939388 29469 raft_consensus.cc:2229] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:55.939428 29469 raft_consensus.cc:2258] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:55.939821 29469 ts_tablet_manager.cc:1905] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.940169 28954 tablet_service.cc:1467] Processing CreateTablet for tablet 6297d07a364c43e995e140b6001f4490 (DEFAULT_TABLE table=test-table [id=6042ebd74e3a4dadae43dbf444ae7d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.940289 28954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297d07a364c43e995e140b6001f4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.940938 29468 ts_tablet_manager.cc:1918] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.940987 29468 log.cc:1196] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/c6d57e2650d04ef08ca0012aae01ce66
I20260808 07:06:55.941248 29468 ts_tablet_manager.cc:1939] T c6d57e2650d04ef08ca0012aae01ce66 P 0b983cba633a49b2b1086fe00c6e081d: Deleting consensus metadata
I20260808 07:06:55.941494 29469 ts_tablet_manager.cc:1918] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.941550 29469 log.cc:1196] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/c6d57e2650d04ef08ca0012aae01ce66
I20260808 07:06:55.941779 29469 ts_tablet_manager.cc:1939] T c6d57e2650d04ef08ca0012aae01ce66 P 634d02277c704f7c8c5c390596ef36a8: Deleting consensus metadata
I20260808 07:06:55.941777 29436 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Bootstrap starting.
I20260808 07:06:55.942121 29436 tablet_bootstrap.cc:654] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.942204 29472 ts_tablet_manager.cc:1905] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:55.942507 29135 tablet_service.cc:1467] Processing CreateTablet for tablet 6297d07a364c43e995e140b6001f4490 (DEFAULT_TABLE table=test-table [id=6042ebd74e3a4dadae43dbf444ae7d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.942621 29135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297d07a364c43e995e140b6001f4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.943691 29472 ts_tablet_manager.cc:1918] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:06:55.943754 29472 log.cc:1196] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/c6d57e2650d04ef08ca0012aae01ce66
I20260808 07:06:55.943830 29437 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Bootstrap starting.
I20260808 07:06:55.944001 29472 ts_tablet_manager.cc:1939] T c6d57e2650d04ef08ca0012aae01ce66 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting consensus metadata
I20260808 07:06:55.944247 29437 tablet_bootstrap.cc:654] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.944677 28750 catalog_manager.cc:4841] TS 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031): tablet c6d57e2650d04ef08ca0012aae01ce66 (table test-table [id=f0829c4615df4d61add9a0675048874b]) successfully deleted
I20260808 07:06:55.944808 28751 catalog_manager.cc:4841] TS a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333): tablet c6d57e2650d04ef08ca0012aae01ce66 (table test-table [id=f0829c4615df4d61add9a0675048874b]) successfully deleted
I20260808 07:06:55.945096 28751 catalog_manager.cc:4841] TS 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159): tablet c6d57e2650d04ef08ca0012aae01ce66 (table test-table [id=f0829c4615df4d61add9a0675048874b]) successfully deleted
I20260808 07:06:55.945155 29297 tablet_service.cc:1467] Processing CreateTablet for tablet 6297d07a364c43e995e140b6001f4490 (DEFAULT_TABLE table=test-table [id=6042ebd74e3a4dadae43dbf444ae7d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:06:55.945254 29297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6297d07a364c43e995e140b6001f4490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:55.945410 29436 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: No bootstrap required, opened a new log
I20260808 07:06:55.945453 29436 ts_tablet_manager.cc:1397] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:06:55.945557 29436 raft_consensus.cc:348] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.945616 29436 raft_consensus.cc:374] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.945637 29436 raft_consensus.cc:729] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6fee4c8cbc34f2a92c39fd6b1d978d0, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.945686 29436 consensus_queue.cc:260] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.945773 29436 ts_tablet_manager.cc:1428] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:55.946156 29437 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: No bootstrap required, opened a new log
I20260808 07:06:55.946198 29437 ts_tablet_manager.cc:1397] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 07:06:55.946328 29437 raft_consensus.cc:348] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.946393 29437 raft_consensus.cc:374] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.946413 29437 raft_consensus.cc:729] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.946414 29435 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Bootstrap starting.
I20260808 07:06:55.946455 29437 consensus_queue.cc:260] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.946538 29437 ts_tablet_manager.cc:1428] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:55.946732 29435 tablet_bootstrap.cc:654] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:55.949927 29435 tablet_bootstrap.cc:492] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: No bootstrap required, opened a new log
I20260808 07:06:55.949991 29435 ts_tablet_manager.cc:1397] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:06:55.950100 29435 raft_consensus.cc:348] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.950178 29435 raft_consensus.cc:374] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:55.950212 29435 raft_consensus.cc:729] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d02277c704f7c8c5c390596ef36a8, State: Initialized, Role: FOLLOWER
I20260808 07:06:55.950260 29435 consensus_queue.cc:260] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:55.950349 29435 ts_tablet_manager.cc:1428] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:56.064481 29443 raft_consensus.cc:482] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:06:56.064616 29443 raft_consensus.cc:504] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:56.064997 29443 leader_election.cc:288] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:56.068163 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.065292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:44617 (local address 127.18.63.65:34333)
0808 07:06:56.065389 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:56.065391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:56.065501 (+ 110us) server_negotiation.cc:407] Connection header received
0808 07:06:56.065562 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:56.065564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:56.065576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:56.065645 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:56.065856 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.066362 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067443 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.067800 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067855 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.067862 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:56.068049 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 07:06:56.068086 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:06:56.068192 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.065525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:35903 (local address 127.18.63.67:39159)
0808 07:06:56.065571 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:56.065573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:56.065634 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:06:56.065690 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:56.065691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:56.065700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:56.065765 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:06:56.065954 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.066449 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067472 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.067817 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067907 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.067915 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:56.068083 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:06:56.068117 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:06:56.068192 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.065195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.18.63.66:44617)
0808 07:06:56.065405 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:56.065418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:56.065495 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:56.065669 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:56.065670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:56.065673 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:56.065766 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.065769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.066458 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.066460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.067344 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.067346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.067897 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.067973 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.067977 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:06:56.068019 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 07:06:56.068079 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":51,"thread_start_us":107,"threads_started":1}
I20260808 07:06:56.068200 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.065446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:39159 (local address 127.18.63.66:35903)
0808 07:06:56.065567 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:56.065577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:56.065633 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:56.065780 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:56.065781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:56.065783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:06:56.065871 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.065874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.066552 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.066553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.067356 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.067358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.067927 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.067928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.067988 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.067991 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:56.068062 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:06:56.068098 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260808 07:06:56.068428 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297d07a364c43e995e140b6001f4490" candidate_uuid: "0b983cba633a49b2b1086fe00c6e081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" is_pre_election: true
I20260808 07:06:56.068531 28979 raft_consensus.cc:2454] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b983cba633a49b2b1086fe00c6e081d in term 0.
I20260808 07:06:56.068701 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297d07a364c43e995e140b6001f4490" candidate_uuid: "0b983cba633a49b2b1086fe00c6e081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8" is_pre_election: true
I20260808 07:06:56.068782 29343 raft_consensus.cc:2454] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b983cba633a49b2b1086fe00c6e081d in term 0.
I20260808 07:06:56.068750 29068 leader_election.cc:302] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, a6fee4c8cbc34f2a92c39fd6b1d978d0; no voters:
I20260808 07:06:56.068897 29443 raft_consensus.cc:2781] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:06:56.068943 29443 raft_consensus.cc:482] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:06:56.068967 29443 raft_consensus.cc:3037] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:56.069790 29443 raft_consensus.cc:504] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:56.069931 29443 leader_election.cc:288] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [CANDIDATE]: Term 1 election: Requested vote from peers a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333), 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159)
I20260808 07:06:56.070233 28979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297d07a364c43e995e140b6001f4490" candidate_uuid: "0b983cba633a49b2b1086fe00c6e081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0"
I20260808 07:06:56.070309 28979 raft_consensus.cc:3037] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:56.070328 29343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6297d07a364c43e995e140b6001f4490" candidate_uuid: "0b983cba633a49b2b1086fe00c6e081d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "634d02277c704f7c8c5c390596ef36a8"
I20260808 07:06:56.070411 29343 raft_consensus.cc:3037] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:56.071102 28979 raft_consensus.cc:2454] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b983cba633a49b2b1086fe00c6e081d in term 1.
I20260808 07:06:56.071187 29343 raft_consensus.cc:2454] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b983cba633a49b2b1086fe00c6e081d in term 1.
I20260808 07:06:56.071362 29068 leader_election.cc:302] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b983cba633a49b2b1086fe00c6e081d, 634d02277c704f7c8c5c390596ef36a8; no voters:
I20260808 07:06:56.071462 29443 raft_consensus.cc:2781] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:56.071614 29443 raft_consensus.cc:686] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 1 LEADER]: Becoming Leader. State: Replica: 0b983cba633a49b2b1086fe00c6e081d, State: Running, Role: LEADER
I20260808 07:06:56.071709 29443 consensus_queue.cc:237] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } }
I20260808 07:06:56.072441 28773 catalog_manager.cc:5495] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66). New cstate: current_term: 1 leader_uuid: "0b983cba633a49b2b1086fe00c6e081d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b983cba633a49b2b1086fe00c6e081d" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 41031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 } health_report { overall_health: UNKNOWN } } }
I20260808 07:06:56.097577 29542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.095238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.18.63.66:41031)
0808 07:06:56.095390 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:56.095392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:56.095400 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:06:56.095448 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:56.095448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:56.095467 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:56.095555 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:56.095673 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.096183 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.096862 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.097058 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.097119 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.097237 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:56.097390 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:56.097468 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:56.097486 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 07:06:56.097505 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":104,"threads_started":1}
I20260808 07:06:56.102643 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.100147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43121 (local address 127.18.63.65:34333)
0808 07:06:56.100224 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:56.100226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:56.100335 (+ 109us) server_negotiation.cc:407] Connection header received
0808 07:06:56.100405 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:56.100406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:56.100417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:56.100488 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:56.100645 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.101098 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.101900 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.102093 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.102184 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.102374 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:56.102413 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:56.102475 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:56.102540 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:06:56.102561 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:56.102685 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.100037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.18.63.66:43121)
0808 07:06:56.100249 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:56.100261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:56.100348 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:56.100491 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:56.100492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:56.100494 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:56.100553 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.100555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.101195 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.101196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.101808 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.101810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.102206 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.102208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.102274 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.102288 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:56.102477 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:56.102480 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:06:56.102534 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:06:56.102584 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260808 07:06:56.104905 29343 raft_consensus.cc:1261] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Refusing update from remote peer 0b983cba633a49b2b1086fe00c6e081d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:56.104919 28979 raft_consensus.cc:1261] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Refusing update from remote peer 0b983cba633a49b2b1086fe00c6e081d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:06:56.105181 29443 consensus_queue.cc:1035] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [LEADER]: Connected to new peer: Peer: permanent_uuid: "634d02277c704f7c8c5c390596ef36a8" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:56.105320 29540 consensus_queue.cc:1035] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6fee4c8cbc34f2a92c39fd6b1d978d0" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:06:56.112360 29542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.110200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:40263 (local address 127.18.63.66:41031)
0808 07:06:56.110255 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:56.110258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:56.110267 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:06:56.110340 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:56.110341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:56.110352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:56.110408 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:06:56.110570 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.111068 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.111735 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.111906 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.111953 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.112101 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:06:56.112144 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:06:56.112195 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:56.112266 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:06:56.112287 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:06:56.112419 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.110164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:41031 (local address 127.18.63.65:40263)
0808 07:06:56.110215 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:56.110226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:56.110267 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:56.110453 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:56.110454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:56.110455 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 07:06:56.110496 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.110498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.111141 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.111142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.111665 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:56.111667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:56.111967 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:56.111967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:56.112006 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:56.112030 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 07:06:56.112211 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 07:06:56.112213 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:06:56.112266 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:06:56.112328 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:06:56.119841 28772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:56.119956 28772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:06:56.120658 28772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c264f7980bba4152811356e30ca26406: Sending DeleteTablet for 3 replicas of tablet 6297d07a364c43e995e140b6001f4490
I20260808 07:06:56.120910 29135 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297d07a364c43e995e140b6001f4490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:56 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:33152
I20260808 07:06:56.120956 28954 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297d07a364c43e995e140b6001f4490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:56 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37766
I20260808 07:06:56.121009 29468 tablet_replica.cc:330] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: stopping tablet replica
I20260808 07:06:56.121048 29472 tablet_replica.cc:330] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: stopping tablet replica
I20260808 07:06:56.121059 29468 raft_consensus.cc:2229] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:06:56.121096 29472 raft_consensus.cc:2229] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:56.121135 29472 raft_consensus.cc:2258] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:56.121136 29468 raft_consensus.cc:2258] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:56.121479 29472 ts_tablet_manager.cc:1905] T 6297d07a364c43e995e140b6001f4490 P a6fee4c8cbc34f2a92c39fd6b1d978d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:56.121601 29468 ts_tablet_manager.cc:1905] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:56.123026 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 28866
I20260808 07:06:56.129406 29297 tablet_service.cc:1514] Processing DeleteTablet for tablet 6297d07a364c43e995e140b6001f4490 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:06:56 UTC) from {username='oryx', principal='oryx/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 07:06:56.129519 29469 tablet_replica.cc:330] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: stopping tablet replica
I20260808 07:06:56.129506 29468 ts_tablet_manager.cc:1918] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:56.129562 29468 log.cc:1196] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/6297d07a364c43e995e140b6001f4490
I20260808 07:06:56.129570 29469 raft_consensus.cc:2229] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:06:56.129611 29469 raft_consensus.cc:2258] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:06:56.129809 29468 ts_tablet_manager.cc:1939] T 6297d07a364c43e995e140b6001f4490 P 0b983cba633a49b2b1086fe00c6e081d: Deleting consensus metadata
I20260808 07:06:56.130115 28750 catalog_manager.cc:4841] TS 0b983cba633a49b2b1086fe00c6e081d (127.18.63.66:41031): tablet 6297d07a364c43e995e140b6001f4490 (table test-table [id=6042ebd74e3a4dadae43dbf444ae7d30]) successfully deleted
I20260808 07:06:56.130409 29469 ts_tablet_manager.cc:1905] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:06:56.132239 29469 ts_tablet_manager.cc:1918] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 07:06:56.132293 29469 log.cc:1196] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/6297d07a364c43e995e140b6001f4490
I20260808 07:06:56.132503 29469 ts_tablet_manager.cc:1939] T 6297d07a364c43e995e140b6001f4490 P 634d02277c704f7c8c5c390596ef36a8: Deleting consensus metadata
I20260808 07:06:56.132978 28751 catalog_manager.cc:4841] TS 634d02277c704f7c8c5c390596ef36a8 (127.18.63.67:39159): tablet 6297d07a364c43e995e140b6001f4490 (table test-table [id=6042ebd74e3a4dadae43dbf444ae7d30]) successfully deleted
W20260808 07:06:56.137620 28751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:34333 (error 108)
I20260808 07:06:56.138159 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 29047
I20260808 07:06:56.138195 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:56.137991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34333 (local address 127.0.0.1:37802)
0808 07:06:56.138092 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:56.138149 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:34333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":60,"threads_started":1}
W20260808 07:06:56.145645 28751 catalog_manager.cc:4584] TS a6fee4c8cbc34f2a92c39fd6b1d978d0 (127.18.63.65:34333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6297d07a364c43e995e140b6001f4490: Network error: Client connection negotiation failed: client connection to 127.18.63.65:34333: connect: Connection refused (error 111)
I20260808 07:06:56.150681 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 29231
I20260808 07:06:56.161144 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 28729
2026-08-08T07:06:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2857 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:56 dist-test-slave-bw88 krb5kdc[29560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:56 dist-test-slave-bw88 krb5kdc[29560](info): set up 2 sockets
Aug 08 07:06:56 dist-test-slave-bw88 krb5kdc[29560](info): commencing operation
krb5kdc: starting...
W20260808 07:06:57.352818 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:57 dist-test-slave-bw88 krb5kdc[29560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:57.475602 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:32853
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:32853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:57.558629 29577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:57.558888 29577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:57.558926 29577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:57.560070 29577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:57.560139 29577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:57.560163 29577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:57.560184 29577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:57.560213 29577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:57.561791 29577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:32853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:32853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:57.562095 29577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:57.562336 29577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:57.567412 29587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:57.567413 29583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:57.567413 29584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:57.567664 29577 server_base.cc:1034] running on GCE node
I20260808 07:06:57.567935 29577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:57.568184 29577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:57.569324 29577 hybrid_clock.cc:648] HybridClock initialized: now 1786172817569310 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T07:06:57Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1405 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:06:57 dist-test-slave-bw88 krb5kdc[29592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:06:57 dist-test-slave-bw88 krb5kdc[29592](info): set up 2 sockets
Aug 08 07:06:57 dist-test-slave-bw88 krb5kdc[29592](info): commencing operation
krb5kdc: starting...
W20260808 07:06:58.755131 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:06:58 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:06:58.878438 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38175
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:38175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:58.960809 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:58.961057 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:58.961095 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:58.962257 29611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:58.962328 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:58.962352 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:58.962374 29611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:58.962402 29611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:58.963881 29611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:38175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:58.964164 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:58.964397 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:58.969208 29624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:58.969369 29622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:58.969269 29619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:58.969470 29611 server_base.cc:1034] running on GCE node
I20260808 07:06:58.969695 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:58.969933 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:58.971099 29611 hybrid_clock.cc:648] HybridClock initialized: now 1786172818971089 us; error 33 us; skew 500 ppm
Aug 08 07:06:58 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172818, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:58.978192 29611 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:58.978678 29611 webserver.cc:460] Webserver started at http://127.18.63.126:44595/ using document root <none> and password file <none>
I20260808 07:06:58.978927 29611 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:58.978995 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:58.979146 29611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:58.980221 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "8740692c29c349579c25e6171be23020"
format_stamp: "Formatted at 2026-08-08 07:06:58 on dist-test-slave-bw88"
server_key: "a15e95fa0d7508e1ed63e53e3d6e6a86"
server_key_iv: "81e94f0641a971426862e4a252d48954"
server_key_version: "encryptionkey@0"
I20260808 07:06:58.980583 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "8740692c29c349579c25e6171be23020"
format_stamp: "Formatted at 2026-08-08 07:06:58 on dist-test-slave-bw88"
server_key: "a15e95fa0d7508e1ed63e53e3d6e6a86"
server_key_iv: "81e94f0641a971426862e4a252d48954"
server_key_version: "encryptionkey@0"
I20260808 07:06:58.982043 29611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:06:58.982815 29633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:58.983009 29611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:06:58.983120 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8740692c29c349579c25e6171be23020"
format_stamp: "Formatted at 2026-08-08 07:06:58 on dist-test-slave-bw88"
server_key: "a15e95fa0d7508e1ed63e53e3d6e6a86"
server_key_iv: "81e94f0641a971426862e4a252d48954"
server_key_version: "encryptionkey@0"
I20260808 07:06:58.983196 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:58.994153 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:58.995967 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:58.996130 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.000495 29611 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:38175
I20260808 07:06:59.000527 29704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:38175 every 8 connection(s)
I20260808 07:06:59.000883 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:59.001844 29705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:06:59.003935 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 29611
I20260808 07:06:59.004057 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:06:59.004289 18685 external_mini_cluster.cc:1428] Setting key 8b74bfd0275f22cbc749cf14174440ac
I20260808 07:06:59.004534 29705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Bootstrap starting.
I20260808 07:06:59.005270 29705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Neither blocks nor log segments found. Creating new log.
I20260808 07:06:59.005640 29705 log.cc:824] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:59.006528 29705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: No bootstrap required, opened a new log
I20260808 07:06:59.008661 29705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.008832 29705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:06:59.008860 29705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8740692c29c349579c25e6171be23020, State: Initialized, Role: FOLLOWER
I20260808 07:06:59.009011 29705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.009084 29705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:59.009109 29705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:59.009176 29705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:06:59.009982 29705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.010105 29705 leader_election.cc:302] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8740692c29c349579c25e6171be23020; no voters:
I20260808 07:06:59.010281 29705 leader_election.cc:288] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:06:59.010347 29712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:06:59.010511 29712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 LEADER]: Becoming Leader. State: Replica: 8740692c29c349579c25e6171be23020, State: Running, Role: LEADER
I20260808 07:06:59.010578 29705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:59.010598 29712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.010915 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8740692c29c349579c25e6171be23020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } } }
I20260808 07:06:59.010993 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:59.011245 29714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8740692c29c349579c25e6171be23020. Latest consensus state: current_term: 1 leader_uuid: "8740692c29c349579c25e6171be23020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } } }
I20260808 07:06:59.011363 29714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:59.011776 29717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:59.012455 29717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:59.014503 29717 catalog_manager.cc:1347] Generated new cluster ID: fb3c5f79c83945609d57e4988a150b5e
I20260808 07:06:59.014559 29717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:59.016439 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.004845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.18.63.126:38175)
0808 07:06:59.005062 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.005072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.005111 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:06:59.005623 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.005631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.005680 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.005836 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.006677 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.007560 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.008724 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.009384 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.009486 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.012513 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:59.012518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:59.012518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:59.012530 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:59.014551 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.014958 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.014961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.014962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.015054 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.015235 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.015236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.015236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.015505 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:59.015636 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:59.016098 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 07:06:59.016230 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260808 07:06:59.022652 29717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:06:59.023207 29717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:59.031863 29717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Generated new TSK 0
I20260808 07:06:59.032051 29717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:06:59.068364 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:59.178886 29738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:59.179088 29738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:59.179112 29738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:59.180289 29738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:59.180333 29738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:59.180359 29738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:59.181895 29738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:59.182149 29738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:59.182400 29738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:59.187783 29743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.187765 29747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.187768 29744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.187987 29738 server_base.cc:1034] running on GCE node
I20260808 07:06:59.188177 29738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:59.188375 29738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:59.189541 29738 hybrid_clock.cc:648] HybridClock initialized: now 1786172819189528 us; error 41 us; skew 500 ppm
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:59.192050 29738 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:06:59.192544 29738 webserver.cc:460] Webserver started at http://127.18.63.65:39731/ using document root <none> and password file <none>
I20260808 07:06:59.192800 29738 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:59.192874 29738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:59.193029 29738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:59.194190 29738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "a9d098a6254b46d99228a3ac8002f008"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "5ba505217a1117efc0c90b824f34044b"
server_key_iv: "3bc64156c59a47c00f12bb0972e263dd"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.194558 29738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "a9d098a6254b46d99228a3ac8002f008"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "5ba505217a1117efc0c90b824f34044b"
server_key_iv: "3bc64156c59a47c00f12bb0972e263dd"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.195921 29738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:06:59.196683 29754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.196863 29738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:59.196974 29738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "a9d098a6254b46d99228a3ac8002f008"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "5ba505217a1117efc0c90b824f34044b"
server_key_iv: "3bc64156c59a47c00f12bb0972e263dd"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.197050 29738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:59.219410 29738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:59.221306 29738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:59.221527 29738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.221793 29738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:59.222131 29738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:59.222189 29738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.222225 29738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:59.222254 29738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.228739 29738 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:36049
I20260808 07:06:59.228803 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:36049 every 8 connection(s)
I20260808 07:06:59.229167 29738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:59.234975 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 29738
I20260808 07:06:59.235107 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:06:59.235342 18685 external_mini_cluster.cc:1428] Setting key 718f2f0b503b3dc5eae321a8651e2e61
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:59.240070 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.230219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:49433 (local address 127.18.63.126:38175)
0808 07:06:59.230290 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.230292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.230522 (+ 230us) server_negotiation.cc:407] Connection header received
0808 07:06:59.231036 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.231037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.231047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.231103 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.232076 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.232586 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.233697 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.233895 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.233951 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.236612 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:59.236613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:59.236614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:59.236623 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:59.238457 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.238981 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.238982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.238982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.239054 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.239347 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.239348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.239348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.239505 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:59.239568 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:59.239905 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 07:06:59.239990 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:06:59.240338 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.230172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:49433)
0808 07:06:59.230358 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:59.230402 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:59.230945 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:59.231170 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:59.231173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:59.231778 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:59.231986 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.231992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.232683 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.232685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.233611 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.233614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.233975 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.233976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.234086 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.234667 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:59.234676 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:59.236512 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:59.238545 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.238554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.238556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.238898 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.239127 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.239128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.239128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.239275 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.239581 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:59.239588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:59.239761 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 07:06:59.239880 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:06:59.240063 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260808 07:06:59.241052 29898 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38175
I20260808 07:06:59.241212 29898 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:59.241544 29898 heartbeater.cc:511] Master 127.18.63.126:38175 requested a full tablet report, sending...
I20260808 07:06:59.242172 29655 ts_manager.cc:194] Registered new tserver with Master: a9d098a6254b46d99228a3ac8002f008 (127.18.63.65:36049)
I20260808 07:06:59.242820 29655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:49433
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:06:59.263980 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:59.349901 29904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:59.350103 29904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:59.350127 29904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:59.351318 29904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:59.351364 29904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:59.351424 29904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:06:59.352890 29904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:59.353153 29904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:59.353420 29904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:59.358516 29913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.358503 29910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.358503 29911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.358708 29904 server_base.cc:1034] running on GCE node
I20260808 07:06:59.358999 29904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:59.359197 29904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:59.360342 29904 hybrid_clock.cc:648] HybridClock initialized: now 1786172819360328 us; error 28 us; skew 500 ppm
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:59.362920 29904 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:06:59.363404 29904 webserver.cc:460] Webserver started at http://127.18.63.66:42469/ using document root <none> and password file <none>
I20260808 07:06:59.363653 29904 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:59.363699 29904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:59.363868 29904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:59.364902 29904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "090d970a869b42079b03321da7402229"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "e32cd572a13ac9d90426a39ddcd82922"
server_key_iv: "c1e537b244fb9b68fdce1933738817e1"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.365268 29904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "090d970a869b42079b03321da7402229"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "e32cd572a13ac9d90426a39ddcd82922"
server_key_iv: "c1e537b244fb9b68fdce1933738817e1"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.366679 29904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:59.367388 29923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.367563 29904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.367690 29904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "090d970a869b42079b03321da7402229"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "e32cd572a13ac9d90426a39ddcd82922"
server_key_iv: "c1e537b244fb9b68fdce1933738817e1"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.367769 29904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:59.378330 29904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:59.380167 29904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:59.380337 29904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.380568 29904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:59.380901 29904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:59.380967 29904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.381009 29904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:59.381038 29904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.387434 29904 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:44653
I20260808 07:06:59.387485 30062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:44653 every 8 connection(s)
I20260808 07:06:59.387791 29904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:06:59.389130 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 29904
I20260808 07:06:59.389216 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:06:59.389542 18685 external_mini_cluster.cc:1428] Setting key c906ff588b10e3f32e0c89b7f6f20308
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:59.398355 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.388484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:55633 (local address 127.18.63.126:38175)
0808 07:06:59.388555 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.388557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.388791 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:06:59.389435 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.389436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.389452 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.389536 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.390637 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.391406 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.392670 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.392921 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.392976 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.395245 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:59.395246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:59.395247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:59.395254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:59.396904 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.397295 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.397297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.397297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.397393 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.397661 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.397663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.397663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.397818 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:59.397898 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:59.398149 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 07:06:59.398233 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:06:59.398363 30067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.388467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:55633)
0808 07:06:59.388655 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:59.388697 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:59.389264 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:59.389612 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:59.389617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:59.390348 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:59.390543 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.390549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.391505 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.391506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.392577 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.392581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.393008 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.393009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.393148 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.393721 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:59.393729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:59.395154 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:59.396995 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.397002 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.397004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.397219 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.397491 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.397492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.397492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.397589 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.397893 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:59.397898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:59.398007 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 07:06:59.398088 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:06:59.398207 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260808 07:06:59.399039 30063 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38175
I20260808 07:06:59.399153 30063 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:59.399345 30063 heartbeater.cc:511] Master 127.18.63.126:38175 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:06:59.399776 29655 ts_manager.cc:194] Registered new tserver with Master: 090d970a869b42079b03321da7402229 (127.18.63.66:44653)
I20260808 07:06:59.400148 29655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:55633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:06:59.419714 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:59.503822 30070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:59.504031 30070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:59.504055 30070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:59.505231 30070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:59.505275 30070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:59.505331 30070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:06:59.506819 30070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:59.507076 30070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:59.507313 30070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:59.512624 30077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.512655 30075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.512693 30081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.512935 30070 server_base.cc:1034] running on GCE node
I20260808 07:06:59.513186 30070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:59.513468 30070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:59.514644 30070 hybrid_clock.cc:648] HybridClock initialized: now 1786172819514624 us; error 44 us; skew 500 ppm
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:59.517443 30070 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:06:59.517868 30070 webserver.cc:460] Webserver started at http://127.18.63.67:38889/ using document root <none> and password file <none>
I20260808 07:06:59.518106 30070 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:59.518184 30070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:59.518333 30070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:06:59.519380 30070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "3ef45b36b0f74b47a734f321ef39f47a"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "b9e05a721e62fb0a3e3e99b476eae910"
server_key_iv: "db701616284e106a4dad2cdf268f6289"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.519753 30070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "3ef45b36b0f74b47a734f321ef39f47a"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "b9e05a721e62fb0a3e3e99b476eae910"
server_key_iv: "db701616284e106a4dad2cdf268f6289"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.521070 30070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:06:59.521919 30088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.522096 30070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.522212 30070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "3ef45b36b0f74b47a734f321ef39f47a"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "b9e05a721e62fb0a3e3e99b476eae910"
server_key_iv: "db701616284e106a4dad2cdf268f6289"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.522296 30070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:59.547554 30070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:59.550051 30070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:59.550227 30070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.550489 30070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:59.550812 30070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:59.550876 30070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.550922 30070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:59.550956 30070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.557178 30070 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:45543
I20260808 07:06:59.557226 30237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:45543 every 8 connection(s)
I20260808 07:06:59.557686 30070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:06:59.565169 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30070
I20260808 07:06:59.565301 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:06:59.565518 18685 external_mini_cluster.cc:1428] Setting key 93ca70583448d1201414b39e5cc0c33a
I20260808 07:06:59.567205 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.558801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:53053 (local address 127.18.63.126:38175)
0808 07:06:59.558861 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.558864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.558896 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:06:59.559416 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.559417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.559427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.559482 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.560433 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.560932 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.562057 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.562252 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.562308 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.564394 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:06:59.564396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:06:59.564396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:06:59.564402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:06:59.565873 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.566387 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.566388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.566388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.566455 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:06:59.566715 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:06:59.566716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:06:59.566716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:06:59.566842 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:06:59.566896 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:06:59.567077 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 07:06:59.567143 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:06:59.567399 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.558532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:53053)
0808 07:06:59.558740 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:59.558811 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:59.559326 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:59.559539 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:59.559542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:59.560154 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:59.560349 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.560354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.561035 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.561036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.561972 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.561975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.562333 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.562334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.562440 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.562875 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:59.562883 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:59.564304 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:06:59.565954 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.565963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.565965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.566308 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.566536 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:06:59.566536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:06:59.566537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:06:59.566656 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:06:59.566886 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:06:59.566888 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:06:59.566993 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 07:06:59.567067 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 07:06:59.567190 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260808 07:06:59.567885 30238 heartbeater.cc:347] Connected to a master server at 127.18.63.126:38175
I20260808 07:06:59.567997 30238 heartbeater.cc:464] Registering TS with master...
I20260808 07:06:59.568190 30238 heartbeater.cc:511] Master 127.18.63.126:38175 requested a full tablet report, sending...
I20260808 07:06:59.568552 29655 ts_manager.cc:194] Registered new tserver with Master: 3ef45b36b0f74b47a734f321ef39f47a (127.18.63.67:45543)
I20260808 07:06:59.568931 29655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:53053
I20260808 07:06:59.577580 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
Entry for principal oryx/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab.
WARNING: no policy specified for oryx/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
Entry for principal oryx/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab.
WARNING: no policy specified for oryx/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
Entry for principal oryx/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab.
I20260808 07:06:59.656152 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 29738
I20260808 07:06:59.667795 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 29904
I20260808 07:06:59.679334 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30070
I20260808 07:06:59.688614 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 29611
I20260808 07:06:59.696605 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38175
--webserver_interface=127.18.63.126
--webserver_port=44595
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:38175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:06:59.779276 30252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:59.779524 30252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:59.779562 30252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:59.780715 30252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:06:59.780793 30252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:59.780819 30252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:59.780853 30252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:06:59.780879 30252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:06:59.782473 30252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:38175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=44595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:59.782753 30252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:59.782992 30252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:59.788115 30259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.788168 30257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.788201 30252 server_base.cc:1034] running on GCE node
W20260808 07:06:59.788116 30263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.788659 30252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:59.788899 30252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:59.790035 30252 hybrid_clock.cc:648] HybridClock initialized: now 1786172819790025 us; error 33 us; skew 500 ppm
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:59.792582 30252 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:06:59.793049 30252 webserver.cc:460] Webserver started at http://127.18.63.126:44595/ using document root <none> and password file <none>
I20260808 07:06:59.793273 30252 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:59.793329 30252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:59.794742 30252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:06:59.795418 30271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.795594 30252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:06:59.795698 30252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8740692c29c349579c25e6171be23020"
format_stamp: "Formatted at 2026-08-08 07:06:58 on dist-test-slave-bw88"
server_key: "a15e95fa0d7508e1ed63e53e3d6e6a86"
server_key_iv: "81e94f0641a971426862e4a252d48954"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.796033 30252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:59.824373 30252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:59.826490 30252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:59.826665 30252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.830955 30252 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:38175
I20260808 07:06:59.831012 30341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:38175 every 8 connection(s)
I20260808 07:06:59.831497 30252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:06:59.832381 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30252
I20260808 07:06:59.832752 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:36049
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:38175
--webserver_port=39731
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:06:59.833683 30342 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:06:59.834610 30342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Bootstrap starting.
I20260808 07:06:59.844117 30342 log.cc:824] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Log is configured to *not* fsync() on all Append() calls
I20260808 07:06:59.846081 30342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:06:59.846383 30342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Bootstrap complete.
I20260808 07:06:59.848712 30342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.848991 30342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8740692c29c349579c25e6171be23020, State: Initialized, Role: FOLLOWER
I20260808 07:06:59.849185 30342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.849267 30342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:06:59.849327 30342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:06:59.849398 30342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:06:59.850409 30342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.850493 30342 leader_election.cc:302] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8740692c29c349579c25e6171be23020; no voters:
I20260808 07:06:59.850700 30342 leader_election.cc:288] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:06:59.850734 30348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:06:59.850883 30348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [term 2 LEADER]: Becoming Leader. State: Replica: 8740692c29c349579c25e6171be23020, State: Running, Role: LEADER
I20260808 07:06:59.850970 30348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } }
I20260808 07:06:59.851001 30342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:06:59.851228 30349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8740692c29c349579c25e6171be23020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } } }
I20260808 07:06:59.851325 30349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:59.851577 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8740692c29c349579c25e6171be23020. Latest consensus state: current_term: 2 leader_uuid: "8740692c29c349579c25e6171be23020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8740692c29c349579c25e6171be23020" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38175 } } }
I20260808 07:06:59.851689 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020 [sys.catalog]: This master's current role is: LEADER
I20260808 07:06:59.851822 30352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:06:59.852284 30352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:06:59.852470 30352 catalog_manager.cc:1259] Loaded cluster ID: fb3c5f79c83945609d57e4988a150b5e
I20260808 07:06:59.852676 30352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:06:59.853914 30352 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:06:59.854096 30352 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8740692c29c349579c25e6171be23020: Loaded TSK: 0
I20260808 07:06:59.854322 30352 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:06:59.920564 30345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:06:59.920792 30345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:06:59.920830 30345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:06:59.922037 30345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:06:59.922102 30345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:06:59.922154 30345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:06:59.923638 30345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.65.keytab
--principal=oryx/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:36049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=39731
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:06:59.923909 30345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:06:59.924157 30345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:06:59.929306 30378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.929386 30383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:06:59.929311 30379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:06:59.929605 30345 server_base.cc:1034] running on GCE node
I20260808 07:06:59.929831 30345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:06:59.930079 30345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:06:59.931248 30345 hybrid_clock.cc:648] HybridClock initialized: now 1786172819931236 us; error 34 us; skew 500 ppm
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172819, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:06:59.934065 30345 init.cc:377] Logged in from keytab as oryx/127.18.63.65@KRBTEST.COM (short username oryx)
I20260808 07:06:59.934635 30345 webserver.cc:460] Webserver started at http://127.18.63.65:39731/ using document root <none> and password file <none>
I20260808 07:06:59.934934 30345 fs_manager.cc:359] Metadata directory not provided
I20260808 07:06:59.935043 30345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:06:59.936902 30345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:06:59.937777 30392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.938006 30345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:06:59.938112 30345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "a9d098a6254b46d99228a3ac8002f008"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "5ba505217a1117efc0c90b824f34044b"
server_key_iv: "3bc64156c59a47c00f12bb0972e263dd"
server_key_version: "encryptionkey@0"
I20260808 07:06:59.938558 30345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:06:59.968783 30345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:06:59.971459 30345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:06:59.971626 30345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:06:59.971899 30345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:06:59.972301 30345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:06:59.972361 30345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.972395 30345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:06:59.972421 30345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:06:59.979841 30345 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:36049
I20260808 07:06:59.979898 30539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:36049 every 8 connection(s)
I20260808 07:06:59.980283 30345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:06:59.980876 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30345
I20260808 07:06:59.981201 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:44653
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:38175
--webserver_port=42469
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:06:59.992726 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.981095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:48549)
0808 07:06:59.981581 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:59.981634 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:59.982408 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:59.983794 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:59.983799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:59.984593 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:59.984880 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.984888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.986924 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.986925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.988231 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.988234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.989155 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.989156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.989300 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.989934 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:59.989944 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:59.992523 (+ 2579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":115,"threads_started":1}
W20260808 07:06:59.992937 30540 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:38175 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.63.126:38175: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
W20260808 07:06:59.993815 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:59.981514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48549 (local address 127.18.63.126:38175)
0808 07:06:59.983239 (+ 1725us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.983249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.983296 (+ 47us) server_negotiation.cc:407] Connection header received
0808 07:06:59.983459 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.983466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.983541 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.983703 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.985067 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.986776 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.988356 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.988948 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.989062 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.993641 (+ 4579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:48549: BlockingRecv error: recv got EOF from 127.18.63.65:48549 (error 108)
Metrics: {"server-negotiator.queue_time_us":1652,"thread_start_us":85,"threads_started":1}
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:06:59 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:06:59.999763 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:06:59.993139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:48913)
0808 07:06:59.994004 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 07:06:59.994015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:06:59.994073 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:06:59.994238 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:06:59.994239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:06:59.994403 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:06:59.994469 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.994471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.995341 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.995342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.996012 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:06:59.996014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.996451 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.996452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:06:59.996511 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.997712 (+ 1201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:06:59.997718 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:06:59.999682 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":89,"threads_started":1}
W20260808 07:06:59.999884 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:06:59.993323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48913 (local address 127.18.63.126:38175)
0808 07:06:59.993944 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 07:06:59.993947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:06:59.994122 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:06:59.994145 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:06:59.994146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:06:59.994158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:06:59.994201 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:06:59.994565 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.995243 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.996108 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:06:59.996345 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:06:59.996400 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:06:59.999813 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:48913: BlockingRecv error: recv got EOF from 127.18.63.65:48913 (error 108)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":136,"threads_started":1}
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:00.004871 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.000308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:60857)
0808 07:07:00.000517 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.000527 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.000579 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.000713 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.000714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.000849 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.000879 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.000880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.001603 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.001604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.002111 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.002113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.002499 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.002500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.002556 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.003031 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.003036 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.004780 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190}
W20260808 07:07:00.004928 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.000424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:60857 (local address 127.18.63.126:38175)
0808 07:07:00.000501 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.000503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.000563 (+ 60us) server_negotiation.cc:407] Connection header received
0808 07:07:00.000638 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.000640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.000651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.000708 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.000961 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.001521 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.002202 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.002408 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.002498 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.004859 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:60857: BlockingRecv error: recv got EOF from 127.18.63.65:60857 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 07:07:00.005170 30540 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:07:00.073454 30545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:00.073661 30545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:00.073685 30545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:00.074851 30545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:00.074893 30545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:00.074920 30545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:07:00.076370 30545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.66.keytab
--principal=oryx/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:44653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=42469
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:00.076607 30545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:00.076843 30545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:00.082057 30553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:00.082172 30554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:00.082098 30556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:00.082278 30545 server_base.cc:1034] running on GCE node
I20260808 07:07:00.082504 30545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:00.082757 30545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:00.083931 30545 hybrid_clock.cc:648] HybridClock initialized: now 1786172820083922 us; error 47 us; skew 500 ppm
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172820, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:00.086766 30545 init.cc:377] Logged in from keytab as oryx/127.18.63.66@KRBTEST.COM (short username oryx)
I20260808 07:07:00.087221 30545 webserver.cc:460] Webserver started at http://127.18.63.66:42469/ using document root <none> and password file <none>
I20260808 07:07:00.087460 30545 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:00.087549 30545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:00.088964 30545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:07:00.089646 30564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.089819 30545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:00.089929 30545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "090d970a869b42079b03321da7402229"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "e32cd572a13ac9d90426a39ddcd82922"
server_key_iv: "c1e537b244fb9b68fdce1933738817e1"
server_key_version: "encryptionkey@0"
I20260808 07:07:00.090299 30545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:00.097570 30545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:00.099956 30545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:00.100129 30545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:00.100371 30545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:00.100683 30545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:00.100740 30545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.100785 30545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:00.100808 30545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.106989 30545 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:44653
I20260808 07:07:00.107039 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:44653 every 8 connection(s)
I20260808 07:07:00.107390 30545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:07:00.108139 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30545
I20260808 07:07:00.108413 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:45543
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:38175
--webserver_port=38889
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:07:00.116154 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.108164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:48461 (local address 127.18.63.126:38175)
0808 07:07:00.108238 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.108241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.108479 (+ 238us) server_negotiation.cc:407] Connection header received
0808 07:07:00.108969 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.108971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.108989 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.109073 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.110220 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.110764 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.112341 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.112530 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.112582 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.116084 (+ 3502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:48461: BlockingRecv error: recv got EOF from 127.18.63.66:48461 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:07:00.116204 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.108141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:48461)
0808 07:07:00.108332 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.108375 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.108871 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.109131 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.109134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.109849 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.110081 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.110090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.110871 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.110872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.112247 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.112252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.112620 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.112622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.112781 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.113439 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.113451 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.115995 (+ 2544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
W20260808 07:07:00.116430 30730 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:38175 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.63.126:38175: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:00.120936 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.116667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:50099)
0808 07:07:00.116718 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.116729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.116768 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.116918 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.116919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.117054 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.117103 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.117104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.117813 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.117814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.118328 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.118329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.118698 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.118699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.118771 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.119194 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.119200 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.120865 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:07:00.121019 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.116742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:50099 (local address 127.18.63.126:38175)
0808 07:07:00.116797 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.116799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.116807 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:07:00.116833 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.116834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.116844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.116914 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.117189 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.117733 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.118415 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.118619 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.118699 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.120946 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:50099: BlockingRecv error: recv got EOF from 127.18.63.66:50099 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:00.125635 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.121209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:39239)
0808 07:07:00.121252 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.121261 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.121293 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.121472 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.121473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.121615 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.121647 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.121648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.122225 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.122226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.123039 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.123041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.123358 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.123359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.123413 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.123801 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.123807 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.125566 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:07:00.125746 30730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:07:00.126008 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.121273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:39239 (local address 127.18.63.126:38175)
0808 07:07:00.121308 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.121310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.121324 (+ 14us) server_negotiation.cc:407] Connection header received
0808 07:07:00.121380 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.121380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.121389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.121447 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.121719 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.122154 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.123117 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.123291 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.123338 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.125959 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:39239: BlockingRecv error: recv got EOF from 127.18.63.66:39239 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:07:00.195057 30734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:00.195261 30734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:00.195286 30734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:00.196450 30734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:00.196494 30734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:00.196552 30734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:07:00.198081 30734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/krb5kdc/oryx_127.18.63.67.keytab
--principal=oryx/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:45543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=38889
--tserver_master_addrs=127.18.63.126:38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:00.198331 30734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:00.198545 30734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:00.203855 30740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:00.203845 30742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:00.203845 30745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:00.204084 30734 server_base.cc:1034] running on GCE node
I20260808 07:07:00.204391 30734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:00.204664 30734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:00.205832 30734 hybrid_clock.cc:648] HybridClock initialized: now 1786172820205828 us; error 39 us; skew 500 ppm
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172820, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:00.208328 30734 init.cc:377] Logged in from keytab as oryx/127.18.63.67@KRBTEST.COM (short username oryx)
I20260808 07:07:00.208776 30734 webserver.cc:460] Webserver started at http://127.18.63.67:38889/ using document root <none> and password file <none>
I20260808 07:07:00.209012 30734 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:00.209097 30734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:00.210636 30734 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:00.211249 30754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.211428 30734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:07:00.211539 30734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "3ef45b36b0f74b47a734f321ef39f47a"
format_stamp: "Formatted at 2026-08-08 07:06:59 on dist-test-slave-bw88"
server_key: "b9e05a721e62fb0a3e3e99b476eae910"
server_key_iv: "db701616284e106a4dad2cdf268f6289"
server_key_version: "encryptionkey@0"
I20260808 07:07:00.211911 30734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:00.220865 30734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:00.222759 30734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:00.222929 30734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:00.223172 30734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:00.223495 30734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:00.223551 30734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.223585 30734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:00.223611 30734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:00.229722 30734 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:45543
I20260808 07:07:00.229779 30895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:45543 every 8 connection(s)
I20260808 07:07:00.230152 30734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:07:00.235092 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30734
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 07:07:00.238664 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.230921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:45663 (local address 127.18.63.126:38175)
0808 07:07:00.230970 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.230972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.231193 (+ 221us) server_negotiation.cc:407] Connection header received
0808 07:07:00.231734 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.231736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.231751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.231803 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.232763 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.233275 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.234334 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.234549 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.234644 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.238579 (+ 3935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:45663: BlockingRecv error: recv got EOF from 127.18.63.67:45663 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:07:00.238741 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.230910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:45663)
0808 07:07:00.231058 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.231094 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.231622 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.231872 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.231876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.232458 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.232659 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.232663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.233409 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.233410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.234235 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.234237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.234659 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.234660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.234794 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.235765 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.235776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.238503 (+ 2727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260808 07:07:00.239059 30896 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:38175 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.63.126:38175: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:00.243439 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.239334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:59575)
0808 07:07:00.239370 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.239380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.239419 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.239556 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.239557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.239692 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.239738 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.239739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.240337 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.240338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.240839 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.240840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.241203 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.241204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.241278 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.241634 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.241638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.243357 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 07:07:00.243494 30544 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.239383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:59575 (local address 127.18.63.126:38175)
0808 07:07:00.239428 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.239429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.239440 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:00.239473 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.239473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.239482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.239552 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.239814 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.240256 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.240921 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.241107 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.241201 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.243429 (+ 2228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:59575: BlockingRecv error: recv got EOF from 127.18.63.67:59575 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:07:00.243782 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.235711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.18.63.126:38175)
0808 07:07:00.235769 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.235771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.235845 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:07:00.236028 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.236030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.236045 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.236099 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.236899 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.237490 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.238651 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.238937 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.239014 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.240303 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:00.240308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:00.240309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:00.240322 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:00.242247 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:00.242638 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:00.242642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:00.242642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:00.242759 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:00.242985 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:00.242986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:00.242987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:00.243277 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:00.243489 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:00.243565 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:07:00.243678 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:00 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:00.248142 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:00.243815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:60547)
0808 07:07:00.243946 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:00.243957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:00.244006 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:00.244387 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:00.244388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:00.244525 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:00.244568 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.244569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.245212 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.245213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.245744 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:00.245745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.246055 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.246055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:00.246092 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.246417 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:00.246421 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:00.248093 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 07:07:00.248224 30547 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:00.243896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:60547 (local address 127.18.63.126:38175)
0808 07:07:00.243948 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:00.243949 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:00.243992 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:07:00.244055 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:00.244056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:00.244245 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:00.244307 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:00.244649 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.245148 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.245813 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:00.245990 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:00.246043 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:00.248184 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:60547: BlockingRecv error: recv got EOF from 127.18.63.67:60547 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 07:07:00.248304 30896 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:01.011483 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:01.005713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:43073)
0808 07:07:01.005900 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:01.005914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:01.005982 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:01.006181 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:01.006182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:01.006362 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:01.006421 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.006423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.007297 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.007299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.008260 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.008264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.008880 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.008881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.008969 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.009443 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:01.009448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:01.011365 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
W20260808 07:07:01.011559 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:01.005816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:43073 (local address 127.18.63.126:38175)
0808 07:07:01.005966 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:01.005968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:01.005978 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:01.006050 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:01.006050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:01.006061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:01.006127 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:01.006503 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.007147 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.008373 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.008790 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.008880 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.011456 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:43073: BlockingRecv error: recv got EOF from 127.18.63.65:43073 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:01.130899 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:01.126091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:42205)
0808 07:07:01.126241 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:01.126251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:01.126307 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:01.126448 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:01.126449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:01.126639 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:01.126695 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.126696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.127643 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.127644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.128193 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.128194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.128514 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.128515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.128557 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.129076 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:01.129081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:01.130832 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
W20260808 07:07:01.130972 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:01.126208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:42205 (local address 127.18.63.126:38175)
0808 07:07:01.126281 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:01.126282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:01.126293 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:01.126369 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:01.126370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:01.126380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:01.126432 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:01.126772 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.127560 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.128267 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.128451 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.128501 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.130923 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:42205: BlockingRecv error: recv got EOF from 127.18.63.66:42205 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:01 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:01.253700 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:01.248696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:48849)
0808 07:07:01.248875 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:01.248885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:01.248942 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:01.249078 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:01.249078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:01.249248 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:01.249301 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.249302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.249998 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.249998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.250555 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:01.250556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.250898 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.250899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:01.250952 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.251531 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:01.251537 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:01.253580 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
W20260808 07:07:01.253726 30901 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:01.248780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:48849 (local address 127.18.63.126:38175)
0808 07:07:01.248864 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:01.248866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:01.248943 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:07:01.248999 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:01.249000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:01.249012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:01.249055 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:01.249436 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.249927 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.250651 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:01.250838 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:01.250884 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:01.253659 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:48849: BlockingRecv error: recv got EOF from 127.18.63.67:48849 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:02.017747 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:02.012086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:46693)
0808 07:07:02.012270 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:02.012287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:02.012371 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:02.012623 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:02.012624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:02.012932 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:02.013006 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.013010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.013793 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.013795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.014378 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.014380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.014774 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.014775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.014844 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.015425 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:02.015431 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:02.017680 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
W20260808 07:07:02.017860 30905 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:02.012226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46693 (local address 127.18.63.126:38175)
0808 07:07:02.012405 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:02.012407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:02.012448 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:07:02.012475 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:02.012475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:02.012505 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:02.012574 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:02.013107 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.013715 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.014491 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.014685 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.014777 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.017780 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:46693: BlockingRecv error: recv got EOF from 127.18.63.65:46693 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:02.136355 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:02.131538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:47641)
0808 07:07:02.131768 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:02.131780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:02.131849 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:02.132003 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:02.132003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:02.132211 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:02.132266 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.132268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.132922 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.132923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.133479 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.133480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.133822 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.133823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.133864 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.134397 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:02.134403 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:02.136294 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":120,"threads_started":1}
W20260808 07:07:02.136529 30905 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:02.131594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:47641 (local address 127.18.63.126:38175)
0808 07:07:02.131669 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:02.131671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:02.131859 (+ 188us) server_negotiation.cc:407] Connection header received
0808 07:07:02.131897 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:02.131899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:02.131912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:02.131995 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:02.132337 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.132848 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.133554 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.133753 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.133807 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.136430 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:47641: BlockingRecv error: recv got EOF from 127.18.63.66:47641 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:02 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:02.259147 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:02.254178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:47135)
0808 07:07:02.254351 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:02.254366 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:02.254435 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:02.254577 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:02.254577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:02.254757 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:02.254810 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.254812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.255462 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.255463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.255999 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:02.256001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.256334 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.256335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:02.256376 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.256897 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:02.256902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:02.259045 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
W20260808 07:07:02.259292 30905 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:02.254325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:47135 (local address 127.18.63.126:38175)
0808 07:07:02.254390 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:02.254392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:02.254419 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:07:02.254501 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:02.254502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:02.254514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:02.254554 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:02.254873 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.255390 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.256073 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:02.256266 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:02.256318 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:02.259226 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:47135: BlockingRecv error: recv got EOF from 127.18.63.67:47135 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:03.023527 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:03.018291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:47183)
0808 07:07:03.018558 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:03.018570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:03.018634 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:03.018783 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:03.018784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:03.018960 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:03.019015 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.019017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.019891 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.019892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.020436 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.020438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.020854 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.020854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.020924 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.021449 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:03.021454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:03.023464 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":196,"threads_started":1}
W20260808 07:07:03.023720 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:03.018495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:47183 (local address 127.18.63.126:38175)
0808 07:07:03.018632 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:03.018634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:03.018644 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:03.018689 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:03.018689 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:03.018700 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:03.018761 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:03.019120 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.019798 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.020525 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.020763 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.020848 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.023638 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:47183: BlockingRecv error: recv got EOF from 127.18.63.65:47183 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:03.141870 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:03.136934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:43149)
0808 07:07:03.137109 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:03.137120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:03.137168 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:03.137372 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:03.137373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:03.137557 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:03.137628 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.137630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.138304 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.138305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.138823 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.138824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.139251 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.139252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.139320 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.139868 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:03.139872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:03.141808 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
W20260808 07:07:03.142014 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:03.137117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43149 (local address 127.18.63.126:38175)
0808 07:07:03.137182 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:03.137184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:03.137195 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:03.137233 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:03.137233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:03.137244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:03.137306 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:03.137707 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.138233 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.138902 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.139129 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.139237 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.141930 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:43149: BlockingRecv error: recv got EOF from 127.18.63.66:43149 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:03 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:03.264154 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:03.259723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:35215)
0808 07:07:03.259913 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:03.259922 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:03.259978 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:03.260085 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:03.260086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:03.260270 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:03.260330 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.260332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.260960 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.260960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.261529 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:03.261531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.261854 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.261855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:03.261897 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.262337 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:03.262341 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:03.264077 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
W20260808 07:07:03.264219 30912 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:03.259770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:35215 (local address 127.18.63.126:38175)
0808 07:07:03.259822 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:03.259825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:03.259970 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:07:03.260021 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:03.260021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:03.260034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:03.260080 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:03.260396 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.260890 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.261595 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:03.261787 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:03.261839 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:03.264172 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:35215: BlockingRecv error: recv got EOF from 127.18.63.67:35215 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:04.060766 30915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:04.023998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:43505)
0808 07:07:04.024191 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:04.024204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:04.024274 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:04.024439 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:04.024440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:04.024628 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:04.024687 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.024689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.025372 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.025374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.025923 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.025925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.026260 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.026261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.026306 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.058660 (+ 32354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:04.058667 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:04.060702 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
W20260808 07:07:04.060923 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:04.024131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:43505 (local address 127.18.63.126:38175)
0808 07:07:04.024261 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:04.024263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:04.024273 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:04.024338 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:04.024338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:04.024348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:04.024407 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:04.024755 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.025259 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.025997 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.026191 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.026243 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.060823 (+ 34580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:43505: BlockingRecv error: recv got EOF from 127.18.63.65:43505 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:04.147198 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:04.142354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:40831)
0808 07:07:04.142536 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:04.142545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:04.142600 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:04.142755 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:04.142756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:04.142963 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:04.143035 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.143037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.143683 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.143684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.144268 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.144270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.144597 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.144598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.144639 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.145068 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:04.145073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:04.147138 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
W20260808 07:07:04.147313 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:04.142428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:40831 (local address 127.18.63.126:38175)
0808 07:07:04.142500 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:04.142502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:04.142598 (+ 96us) server_negotiation.cc:407] Connection header received
0808 07:07:04.142675 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:04.142676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:04.142687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:04.142729 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:04.143104 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.143600 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.144343 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.144531 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.144585 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.147237 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:40831: BlockingRecv error: recv got EOF from 127.18.63.66:40831 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:04 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:04.269227 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:04.264685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:45035)
0808 07:07:04.264834 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:04.264845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:04.264900 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:04.265045 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:04.265046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:04.265241 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:04.265294 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.265296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.265953 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.265954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.266501 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:04.266502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.266839 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.266840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:04.266884 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.267396 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:04.267401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:04.269169 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
W20260808 07:07:04.269421 30917 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:04.264781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:45035 (local address 127.18.63.126:38175)
0808 07:07:04.264847 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:04.264849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:04.264888 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:07:04.264957 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:04.264957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:04.264969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:04.265024 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:04.265413 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.265886 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.266578 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:04.266771 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:04.266827 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:04.269311 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:45035: BlockingRecv error: recv got EOF from 127.18.63.67:45035 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:05.066054 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:05.061357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:45845)
0808 07:07:05.061515 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:05.061526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:05.061584 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:05.061744 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:05.061745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:05.061919 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:05.061974 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.061976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.062625 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.062626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.063163 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.063164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.063488 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.063489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.063531 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.064097 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:05.064103 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:05.065992 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
W20260808 07:07:05.066190 30922 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:05.061463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:45845 (local address 127.18.63.126:38175)
0808 07:07:05.061602 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:05.061604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:05.061612 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:07:05.061651 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:05.061652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:05.061662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:05.061722 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:05.062043 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.062539 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.063234 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.063422 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.063473 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.066102 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:45845: BlockingRecv error: recv got EOF from 127.18.63.65:45845 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:05.152621 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:05.147765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:51239)
0808 07:07:05.147950 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:05.147970 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:05.148029 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:05.148179 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:05.148180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:05.148352 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:05.148406 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.148408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.149044 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.149045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.149639 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.149641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.149956 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.149956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.149997 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.150506 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:05.150511 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:05.152560 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":64,"threads_started":1}
W20260808 07:07:05.152746 30922 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:05.147902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:51239 (local address 127.18.63.126:38175)
0808 07:07:05.147972 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:05.147974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:05.148013 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:07:05.148089 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:05.148090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:05.148101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:05.148154 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:05.148470 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.148978 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.149709 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.149894 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.149944 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.152670 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:51239: BlockingRecv error: recv got EOF from 127.18.63.66:51239 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:05 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:05.274603 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:05.269707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:59933)
0808 07:07:05.269897 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:05.269910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:05.270036 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:05.270236 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:05.270237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:05.270434 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:05.270492 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.270495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.271214 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.271216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.271756 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:05.271758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.272143 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.272145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:05.272208 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.272650 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:05.272655 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:05.274502 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
W20260808 07:07:05.274659 30922 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:05.269801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:59933 (local address 127.18.63.126:38175)
0808 07:07:05.269889 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:05.269891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:05.270012 (+ 121us) server_negotiation.cc:407] Connection header received
0808 07:07:05.270159 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:05.270160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:05.270175 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:05.270219 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:05.270583 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.271115 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.271867 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:05.272068 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:05.272144 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:05.274590 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:59933: BlockingRecv error: recv got EOF from 127.18.63.67:59933 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:06.071471 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:06.066607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:41851)
0808 07:07:06.066807 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:06.066824 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:06.066898 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:06.067052 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:06.067053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:06.067238 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:06.067291 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.067293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.067926 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.067927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.068455 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.068456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.068783 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.068784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.068826 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.069283 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:06.069288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:06.071410 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":82,"threads_started":1}
W20260808 07:07:06.071594 30927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:06.066781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:41851 (local address 127.18.63.126:38175)
0808 07:07:06.066922 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:06.066924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:06.066932 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:07:06.066955 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:06.066956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:06.066966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:06.067023 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:06.067359 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.067855 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.068528 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.068717 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.068767 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.071510 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:41851: BlockingRecv error: recv got EOF from 127.18.63.65:41851 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:06.158267 30928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:06.153132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:55499)
0808 07:07:06.153320 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:06.153358 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:06.153444 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:06.153630 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:06.153631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:06.153833 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:06.153884 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.153886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.154580 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.154581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.155132 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.155134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.155521 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.155522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.155583 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.156223 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:06.156230 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:06.158205 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
W20260808 07:07:06.158376 30927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:06.153216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:55499 (local address 127.18.63.126:38175)
0808 07:07:06.153306 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:06.153309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:06.153425 (+ 116us) server_negotiation.cc:407] Connection header received
0808 07:07:06.153525 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:06.153526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:06.153541 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:06.153619 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:06.153969 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.154489 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.155237 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.155443 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.155520 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.158304 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:55499: BlockingRecv error: recv got EOF from 127.18.63.66:55499 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:06 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:06.280056 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:06.275207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:57595)
0808 07:07:06.275427 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:06.275437 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:06.275516 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:06.275685 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:06.275686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:06.275875 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:06.275926 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.275928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.276608 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.276609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.277157 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:06.277159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.277617 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.277618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:06.277681 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.278151 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:06.278156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:06.279996 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
W20260808 07:07:06.280194 30927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:06.275273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:57595 (local address 127.18.63.126:38175)
0808 07:07:06.275330 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:06.275333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:06.275499 (+ 166us) server_negotiation.cc:407] Connection header received
0808 07:07:06.275573 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:06.275574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:06.275589 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:06.275665 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:06.276008 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.276515 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.277245 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:06.277540 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:06.277625 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:06.280106 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:57595: BlockingRecv error: recv got EOF from 127.18.63.67:57595 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:07.076592 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:07.072012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:56065)
0808 07:07:07.072164 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:07.072176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:07.072242 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:07.072391 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:07.072391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:07.072565 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:07.072636 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.072638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.073275 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.073275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.073837 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.073839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.074167 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.074168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.074210 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.074669 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:07.074674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:07.076529 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":110,"threads_started":1}
W20260808 07:07:07.076742 30931 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:07.072108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:56065 (local address 127.18.63.126:38175)
0808 07:07:07.072265 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:07.072267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:07.072274 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:07:07.072296 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:07.072297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:07.072307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:07.072370 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:07.072708 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.073206 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.073908 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.074100 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.074151 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.076648 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:56065: BlockingRecv error: recv got EOF from 127.18.63.65:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:07.163358 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:07.158799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:39475)
0808 07:07:07.158947 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:07.158958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:07.159017 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:07.159192 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:07.159193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:07.159366 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:07.159421 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.159422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.160085 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.160086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.160601 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.160602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.160963 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.160964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.161007 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.161565 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:07.161570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:07.163297 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
W20260808 07:07:07.163477 30931 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:07.158877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:39475 (local address 127.18.63.126:38175)
0808 07:07:07.159074 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:07.159076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:07.159085 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:07:07.159110 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:07.159111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:07.159123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:07.159173 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:07.159508 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.160014 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.160690 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.160893 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.160950 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.163398 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:39475: BlockingRecv error: recv got EOF from 127.18.63.66:39475 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:07 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:07.285504 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:07.280589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:45189)
0808 07:07:07.280859 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:07.280873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:07.280950 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:07.281088 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:07.281088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:07.281276 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:07.281330 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.281357 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.282032 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.282033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.282687 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:07.282690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.283167 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.283168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:07.283219 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.283667 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:07.283671 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:07.285443 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
W20260808 07:07:07.285651 30931 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:07.280807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:45189 (local address 127.18.63.126:38175)
0808 07:07:07.280885 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:07.280887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:07.280945 (+ 58us) server_negotiation.cc:407] Connection header received
0808 07:07:07.280999 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:07.281000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:07.281012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:07.281061 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:07.281448 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.281944 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.282779 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:07.283085 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:07.283153 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:07.285555 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:45189: BlockingRecv error: recv got EOF from 127.18.63.67:45189 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:08.081903 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:08.077120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:54485)
0808 07:07:08.077282 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:08.077292 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:08.077386 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:08.077568 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:08.077569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:08.077742 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:08.077796 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.077797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.078450 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.078450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.078970 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.078971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.079306 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.079307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.079349 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.079837 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:08.079842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:08.081789 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
W20260808 07:07:08.081949 30935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:08.077279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:54485 (local address 127.18.63.126:38175)
0808 07:07:08.077440 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:08.077441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:08.077449 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:07:08.077472 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:08.077473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:08.077483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:08.077547 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:08.077865 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.078381 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.079041 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.079239 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.079288 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.081873 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:54485: BlockingRecv error: recv got EOF from 127.18.63.65:54485 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:08.168869 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:08.163935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:53849)
0808 07:07:08.164062 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:08.164075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:08.164155 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:08.164287 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:08.164287 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:08.164462 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:08.164521 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.164523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.165301 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.165302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.165853 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.165855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.166208 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.166210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.166252 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.166764 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:08.166769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:08.168798 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260808 07:07:08.168998 30935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:08.164019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:53849 (local address 127.18.63.126:38175)
0808 07:07:08.164129 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:08.164131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:08.164142 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:08.164207 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:08.164207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:08.164219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:08.164272 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:08.164591 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.165231 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.165937 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.166137 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.166197 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.168893 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:53849: BlockingRecv error: recv got EOF from 127.18.63.66:53849 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:08 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:08.291004 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:08.286103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:40103)
0808 07:07:08.286243 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:08.286258 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:08.286335 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:08.286493 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:08.286494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:08.286692 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:08.286746 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.286747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.287518 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.287520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.288061 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:08.288062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.288435 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.288436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:08.288496 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.289009 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:08.289014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:08.290935 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
W20260808 07:07:08.291098 30935 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:08.286224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:40103 (local address 127.18.63.126:38175)
0808 07:07:08.286276 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:08.286279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:08.286313 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:07:08.286405 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:08.286410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:08.286421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:08.286486 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:08.286835 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.287422 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.288168 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:08.288360 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:08.288437 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:08.291031 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:40103: BlockingRecv error: recv got EOF from 127.18.63.67:40103 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:09.091699 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:09.082404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:33467)
0808 07:07:09.082552 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:09.082564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:09.082632 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:09.082897 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:09.082897 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:09.083078 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:09.083134 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.083136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.083839 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.083840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.084364 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.084365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.084709 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.084711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.084771 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.089297 (+ 4526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:09.089308 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:09.091599 (+ 2291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
W20260808 07:07:09.091820 30941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:09.082596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:33467 (local address 127.18.63.126:38175)
0808 07:07:09.082752 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:09.082754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:09.082764 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:09.082797 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:09.082798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:09.082810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:09.082877 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:09.083213 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.083768 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.084437 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.084632 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.084682 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.091727 (+ 7045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:33467: BlockingRecv error: recv got EOF from 127.18.63.65:33467 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:09.174377 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:09.169406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:45935)
0808 07:07:09.169578 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:09.169590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:09.169644 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:09.169814 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:09.169814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:09.170012 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:09.170072 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.170074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.170723 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.170725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.171351 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.171353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.171719 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.171720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.171776 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.172242 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:09.172247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:09.174316 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
W20260808 07:07:09.174521 30941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:09.169599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:45935 (local address 127.18.63.126:38175)
0808 07:07:09.169671 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:09.169673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:09.169682 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:07:09.169717 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:09.169718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:09.169729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:09.169787 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:09.170144 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.170637 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.171439 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.171629 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.171685 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.174437 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:45935: BlockingRecv error: recv got EOF from 127.18.63.66:45935 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:09 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:09.296592 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:09.291536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:60953)
0808 07:07:09.291704 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:09.291721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:09.291780 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:09.291893 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:09.291893 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:09.292068 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:09.292124 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.292126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.292840 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.292841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.293577 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:09.293581 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.293982 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.293983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:09.294034 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.294546 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:09.294551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:09.296512 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260808 07:07:09.296747 30941 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:09.291620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:60953 (local address 127.18.63.126:38175)
0808 07:07:09.291675 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:09.291677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:09.291781 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:07:09.291825 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:09.291826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:09.291837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:09.291888 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:09.292186 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.292693 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.293660 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:09.293853 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:09.293983 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:09.296657 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:60953: BlockingRecv error: recv got EOF from 127.18.63.67:60953 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:10.097152 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:10.092352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:38145)
0808 07:07:10.092536 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:10.092548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:10.092605 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:10.092797 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:10.092798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:10.093027 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:10.093141 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.093144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.093932 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.093933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.094477 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.094478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.094854 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.094855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.094897 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.095347 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:10.095351 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:10.097062 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
W20260808 07:07:10.097250 30945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:10.092456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:38145 (local address 127.18.63.126:38175)
0808 07:07:10.092572 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:10.092576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:10.092592 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:07:10.092676 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:10.092677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:10.092687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:10.092747 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:10.093213 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.093854 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.094565 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.094785 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.094839 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.097175 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:38145: BlockingRecv error: recv got EOF from 127.18.63.65:38145 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:10.180137 30947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:10.174979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:43451)
0808 07:07:10.175145 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:10.175157 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:10.175227 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:10.175413 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:10.175414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:10.175621 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:10.175674 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.175676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.176376 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.176377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.176919 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.176920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.177310 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.177310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.177409 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.177947 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:10.177951 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:10.180061 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
W20260808 07:07:10.180238 30945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:10.175042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43451 (local address 127.18.63.126:38175)
0808 07:07:10.175144 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:10.175147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:10.175209 (+ 62us) server_negotiation.cc:407] Connection header received
0808 07:07:10.175323 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:10.175324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:10.175338 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:10.175399 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:10.175762 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.176279 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.177009 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.177231 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.177311 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.180170 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:43451: BlockingRecv error: recv got EOF from 127.18.63.66:43451 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:10 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:10.302124 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:10.297144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:37613)
0808 07:07:10.297295 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:10.297306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:10.297420 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:10.297597 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:10.297598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:10.297773 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:10.297828 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.297829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.298694 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.298695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.299320 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:10.299322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.299700 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.299701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:10.299760 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.300204 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:10.300208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:10.302052 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
W20260808 07:07:10.302227 30945 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:10.297273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:37613 (local address 127.18.63.126:38175)
0808 07:07:10.297389 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:10.297391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:10.297402 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:10.297506 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:10.297507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:10.297519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:10.297576 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:10.297890 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.298592 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.299387 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:10.299633 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:10.299690 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:10.302154 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:37613: BlockingRecv error: recv got EOF from 127.18.63.67:37613 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:11.102310 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:11.097641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:46055)
0808 07:07:11.097797 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:11.097809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:11.097877 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:11.098005 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:11.098006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:11.098193 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:11.098245 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.098247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.098976 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.098977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.099536 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.099538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.099940 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.099941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.099983 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.100416 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:11.100421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:11.102238 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260808 07:07:11.102447 30950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:11.097720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46055 (local address 127.18.63.126:38175)
0808 07:07:11.097833 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:11.097835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:11.097858 (+ 23us) server_negotiation.cc:407] Connection header received
0808 07:07:11.097924 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:11.097925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:11.097935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:11.097993 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:11.098308 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.098907 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.099606 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.099871 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.099924 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.102335 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:46055: BlockingRecv error: recv got EOF from 127.18.63.65:46055 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":57,"threads_started":1}
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:11.185304 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:11.180678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:53971)
0808 07:07:11.180836 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:11.180848 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:11.180915 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:11.181121 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:11.181122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:11.181317 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:11.181398 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.181400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.182070 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.182070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.182620 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.182621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.182946 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.182947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.182996 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.183448 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:11.183452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:11.185233 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
W20260808 07:07:11.185474 30950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:11.180766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:53971 (local address 127.18.63.126:38175)
0808 07:07:11.180857 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:11.180859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:11.180905 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:07:11.180963 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:11.180964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:11.180980 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:11.181041 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:11.181495 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.181996 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.182697 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.182881 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.182931 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.185331 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:53971: BlockingRecv error: recv got EOF from 127.18.63.66:53971 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:11 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:11.307219 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:11.302647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:52987)
0808 07:07:11.302938 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:11.302949 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:11.303008 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:11.303149 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:11.303149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:11.303326 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:11.303377 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.303379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.304014 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.304015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.304560 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:11.304562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.304887 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.304888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:11.304930 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.305434 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:11.305439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:11.307149 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":99,"threads_started":1}
W20260808 07:07:11.307283 30950 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:11.302815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52987 (local address 127.18.63.126:38175)
0808 07:07:11.302887 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:11.302889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:11.303008 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:07:11.303051 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:11.303052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:11.303065 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:11.303119 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:11.303445 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.303946 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.304639 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:11.304820 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:11.304870 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:11.307233 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:52987: BlockingRecv error: recv got EOF from 127.18.63.67:52987 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:12.110517 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:12.102789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:42645)
0808 07:07:12.102945 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:12.102956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:12.103006 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:12.103160 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:12.103161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:12.103341 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:12.103393 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.103395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.104099 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.104100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.104632 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.104633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.104985 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.104986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.105027 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.108593 (+ 3566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:12.108599 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:12.110459 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260808 07:07:12.110674 30955 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:12.102855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:42645 (local address 127.18.63.126:38175)
0808 07:07:12.103024 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:12.103025 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:12.103036 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:12.103059 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:12.103059 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:12.103070 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:12.103143 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:12.103499 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.104023 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.104712 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.104908 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.104975 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.110587 (+ 5612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:42645: BlockingRecv error: recv got EOF from 127.18.63.65:42645 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:12.190578 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:12.185986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:44307)
0808 07:07:12.186159 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:12.186171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:12.186244 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:12.186391 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:12.186392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:12.186572 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:12.186627 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.186629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.187273 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.187274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.187815 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.187816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.188145 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.188146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.188188 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.188639 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:12.188643 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:12.190518 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260808 07:07:12.190716 30955 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:12.186101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:44307 (local address 127.18.63.126:38175)
0808 07:07:12.186175 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:12.186178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:12.186246 (+ 68us) server_negotiation.cc:407] Connection header received
0808 07:07:12.186308 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:12.186309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:12.186320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:12.186364 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:12.186692 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.187193 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.187893 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.188075 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.188134 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.190615 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:44307: BlockingRecv error: recv got EOF from 127.18.63.66:44307 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:12 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:12.312644 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:12.307805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:46651)
0808 07:07:12.307995 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:12.308008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:12.308104 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:12.308306 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:12.308307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:12.308509 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:12.308559 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.308561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.309258 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.309260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.309866 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:12.309868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.310230 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.310231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:12.310294 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.310754 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:12.310758 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:12.312557 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
W20260808 07:07:12.312757 30955 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:12.307918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:46651 (local address 127.18.63.126:38175)
0808 07:07:12.307999 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:12.308001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:12.308084 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:07:12.308212 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:12.308213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:12.308233 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:12.308298 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:12.308660 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.309170 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.309955 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:12.310153 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:12.310230 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:12.312684 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:46651: BlockingRecv error: recv got EOF from 127.18.63.67:46651 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:13.115885 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:13.111003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:32891)
0808 07:07:13.111152 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:13.111162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:13.111210 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:13.111378 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:13.111379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:13.111552 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:13.111604 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.111620 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.112248 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.112248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.112800 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.112802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.113252 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.113253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.113322 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.113814 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:13.113819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:13.115816 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260808 07:07:13.116047 30959 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:13.111058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:32891 (local address 127.18.63.126:38175)
0808 07:07:13.111185 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:13.111187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:13.111198 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:07:13.111274 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:13.111275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:13.111287 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:13.111354 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:13.111679 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.112180 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.112898 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.113159 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.113234 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.115967 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:32891: BlockingRecv error: recv got EOF from 127.18.63.65:32891 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:13.196017 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:13.191267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:50657)
0808 07:07:13.191452 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:13.191464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:13.191516 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:13.191666 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:13.191666 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:13.191847 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:13.191900 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.191901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.192537 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.192538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.193211 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.193214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.193569 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.193570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.193626 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.194100 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:13.194104 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:13.195946 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":106,"threads_started":1}
W20260808 07:07:13.196130 30959 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:13.191409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:50657 (local address 127.18.63.126:38175)
0808 07:07:13.191485 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:13.191487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:13.191499 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:07:13.191579 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:13.191580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:13.191591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:13.191641 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:13.191963 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.192484 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.193286 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.193495 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.193548 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.196047 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:50657: BlockingRecv error: recv got EOF from 127.18.63.66:50657 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:13 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:13.318163 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:13.313149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:47457)
0808 07:07:13.313369 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:13.313382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:13.313476 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:13.313651 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:13.313653 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:13.313848 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:13.313898 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.313900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.314662 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.314663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.315242 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:13.315244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.315694 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.315695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:13.315768 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.316203 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:13.316207 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:13.318044 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
W20260808 07:07:13.318197 30959 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:13.313254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:47457 (local address 127.18.63.126:38175)
0808 07:07:13.313361 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:13.313363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:13.313456 (+ 93us) server_negotiation.cc:407] Connection header received
0808 07:07:13.313534 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:13.313535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:13.313549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:13.313637 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:13.313983 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.314550 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.315332 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:13.315600 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:13.315690 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:13.318127 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:47457: BlockingRecv error: recv got EOF from 127.18.63.67:47457 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:14.126279 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:14.116380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:37147)
0808 07:07:14.116571 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:14.116585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:14.116659 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:14.116808 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:14.116809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:14.117016 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:14.117069 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.117070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.117941 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.117942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.118476 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.118478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.118818 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.118819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.118862 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.124394 (+ 5532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:14.124399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:14.126218 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":63,"threads_started":1}
W20260808 07:07:14.126411 30965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:14.116456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:37147 (local address 127.18.63.126:38175)
0808 07:07:14.116584 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:14.116586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:14.116651 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:07:14.116703 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:14.116704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:14.116715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:14.116788 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:14.117131 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.117778 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.118554 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.118750 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.118800 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.126324 (+ 7524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:37147: BlockingRecv error: recv got EOF from 127.18.63.65:37147 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:14.201668 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:14.196626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:59671)
0808 07:07:14.196786 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:14.196799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:14.196871 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:14.197095 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:14.197096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:14.197327 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:14.197406 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.197408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.198156 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.198157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.198731 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.198732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.199164 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.199165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.199208 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.199699 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:14.199704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:14.201603 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
W20260808 07:07:14.201817 30965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:14.196704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:59671 (local address 127.18.63.126:38175)
0808 07:07:14.196774 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:14.196777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:14.196863 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:07:14.196974 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:14.196975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:14.196992 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:14.197077 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:14.197494 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.198057 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.198844 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.199072 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.199171 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.201732 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:59671: BlockingRecv error: recv got EOF from 127.18.63.66:59671 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:14 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.67@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:14.323225 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:14.318673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.67:60271)
0808 07:07:14.318861 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:14.318873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:14.318937 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:14.319070 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:14.319071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:14.319258 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:14.319310 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.319311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.319929 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.319929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.320476 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:14.320477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.320785 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.320786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:14.320827 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.321290 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:14.321295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:14.323166 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
W20260808 07:07:14.323336 30965 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:14.318766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:60271 (local address 127.18.63.126:38175)
0808 07:07:14.318833 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:14.318835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:14.318939 (+ 104us) server_negotiation.cc:407] Connection header received
0808 07:07:14.318980 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:14.318981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:14.318994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:14.319047 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:14.319376 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.319863 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.320545 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:14.320722 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:14.320770 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:14.323264 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.67:60271: BlockingRecv error: recv got EOF from 127.18.63.67:60271 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 07:07:15 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:15 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.65@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:15.131614 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.126949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.65:48607)
0808 07:07:15.127101 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:15.127113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:15.127163 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:15.127300 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:15.127301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:15.127470 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:15.127524 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.127555 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.128235 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.128235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.128786 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.128787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.129114 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.129115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.129158 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.129699 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:15.129704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:15.131543 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
W20260808 07:07:15.131721 30971 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:15.127035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48607 (local address 127.18.63.126:38175)
0808 07:07:15.127168 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.127170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.127179 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:07:15.127202 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.127202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.127212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.127279 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.127637 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.128165 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.128857 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.129049 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.129099 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.131645 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.65:48607: BlockingRecv error: recv got EOF from 127.18.63.65:48607 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
Aug 08 07:07:15 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 07:07:15 dist-test-slave-bw88 krb5kdc[29592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.63.66@KRBTEST.COM for oryx/127.18.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 07:07:15.206540 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.202164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:38175 (local address 127.18.63.66:35147)
0808 07:07:15.202344 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:15.202353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:15.202407 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:15.202541 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:15.202542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:15.202708 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:15.202762 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.202763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.203386 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.203387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.203917 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.203919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.204246 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.204247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.204287 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.204728 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:15.204733 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:15.206463 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.63.126:38175: Server oryx/127.18.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
W20260808 07:07:15.206687 30971 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:15.202246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:35147 (local address 127.18.63.126:38175)
0808 07:07:15.202315 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.202317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.202409 (+ 92us) server_negotiation.cc:407] Connection header received
0808 07:07:15.202455 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.202456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.202467 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.202522 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.202825 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.203317 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.203992 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.204182 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.204233 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.206604 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.63.66:35147: BlockingRecv error: recv got EOF from 127.18.63.66:35147 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:07:15.240677 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30345
I20260808 07:07:15.249971 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30545
I20260808 07:07:15.269495 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30734
I20260808 07:07:15.278451 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30252
2026-08-08T07:07:15Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17710 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T07:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:07:15Z Disabled control of system clock
I20260808 07:07:15.312777 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44677
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:44677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 07:07:15.395285 30978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:15.395488 30978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:15.395509 30978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:15.396600 30978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:07:15.396637 30978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:15.396651 30978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:15.396662 30978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:07:15.396674 30978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:07:15.398089 30978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:44677
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:15.398355 30978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:15.398666 30978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:15.403903 30984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.403908 30985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.403908 30987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:15.404203 30978 server_base.cc:1034] running on GCE node
I20260808 07:07:15.404614 30978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:15.404945 30978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:15.406119 30978 hybrid_clock.cc:648] HybridClock initialized: now 1786172835406092 us; error 39 us; skew 500 ppm
I20260808 07:07:15.407362 30978 webserver.cc:460] Webserver started at http://127.18.63.126:36467/ using document root <none> and password file <none>
I20260808 07:07:15.407689 30978 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:15.407764 30978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:15.407920 30978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:15.408993 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "b3a222b1830a4b238b3cc80680867245"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6b45cf0408a4b0a0b18486c6aa26f7be"
server_key_iv: "4bd1675c5c76576d6fc1c042b3f73c1b"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.409415 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "b3a222b1830a4b238b3cc80680867245"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6b45cf0408a4b0a0b18486c6aa26f7be"
server_key_iv: "4bd1675c5c76576d6fc1c042b3f73c1b"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.410741 30978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:07:15.411464 30994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.411648 30978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:07:15.411752 30978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "b3a222b1830a4b238b3cc80680867245"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6b45cf0408a4b0a0b18486c6aa26f7be"
server_key_iv: "4bd1675c5c76576d6fc1c042b3f73c1b"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.411835 30978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:15.426985 30978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:15.427301 30978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:15.427466 30978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:15.431514 30978 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:44677
I20260808 07:07:15.431555 31061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:44677 every 8 connection(s)
I20260808 07:07:15.431901 30978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:07:15.432632 31063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:15.435283 31063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245: Bootstrap starting.
I20260808 07:07:15.435845 31063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:15.436101 31063 log.cc:824] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:15.436846 31063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245: No bootstrap required, opened a new log
I20260808 07:07:15.437412 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 30978
I20260808 07:07:15.437520 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:07:15.437729 18685 external_mini_cluster.cc:1428] Setting key 416fe52e228e9a8a9baeacec800cdd94
I20260808 07:07:15.439196 31063 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } }
I20260808 07:07:15.439374 31063 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:15.439417 31063 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a222b1830a4b238b3cc80680867245, State: Initialized, Role: FOLLOWER
I20260808 07:07:15.439574 31063 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } }
I20260808 07:07:15.439684 31063 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:07:15.439730 31063 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:07:15.439779 31063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:15.440325 31063 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } }
I20260808 07:07:15.440457 31063 leader_election.cc:302] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3a222b1830a4b238b3cc80680867245; no voters:
I20260808 07:07:15.440668 31063 leader_election.cc:288] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:07:15.440701 31069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:07:15.440815 31069 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [term 1 LEADER]: Becoming Leader. State: Replica: b3a222b1830a4b238b3cc80680867245, State: Running, Role: LEADER
I20260808 07:07:15.440960 31069 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } }
I20260808 07:07:15.440968 31063 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:07:15.441907 31071 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3a222b1830a4b238b3cc80680867245. Latest consensus state: current_term: 1 leader_uuid: "b3a222b1830a4b238b3cc80680867245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } } }
I20260808 07:07:15.442001 31071 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:15.442353 31084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:07:15.442880 31084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:07:15.442943 31070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3a222b1830a4b238b3cc80680867245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3a222b1830a4b238b3cc80680867245" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44677 } } }
I20260808 07:07:15.443135 31070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245 [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:15.445156 31084 catalog_manager.cc:1347] Generated new cluster ID: 52aae5460ff8421388f5806674e6828e
I20260808 07:07:15.445189 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.438286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.18.63.126:44677)
0808 07:07:15.438542 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.438551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.438587 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:07:15.439144 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.439153 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.439235 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.439383 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.439643 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.441087 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.442599 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.443340 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.443967 (+ 627us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.444316 (+ 349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:15.444320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:15.444321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:15.444554 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:15.444599 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:15.444614 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:15.444694 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:15.444878 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:07:15.444960 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260808 07:07:15.445227 31084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:07:15.486850 31084 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:07:15.487500 31084 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:07:15.493139 31084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3a222b1830a4b238b3cc80680867245: Generated new TSK 0
I20260808 07:07:15.493363 31084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:07:15.506543 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--builtin_ntp_servers=127.18.63.84:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 07:07:15.597318 31094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:15.597553 31094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:15.597573 31094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:15.598702 31094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:15.598743 31094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:15.598766 31094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:07:15.600216 31094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:15.600459 31094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:15.600759 31094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:07:15.606132 31094 server_base.cc:1034] running on GCE node
W20260808 07:07:15.606096 31100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.606086 31104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.606096 31101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:15.606592 31094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:15.606844 31094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:15.608011 31094 hybrid_clock.cc:648] HybridClock initialized: now 1786172835607996 us; error 27 us; skew 500 ppm
I20260808 07:07:15.609277 31094 webserver.cc:460] Webserver started at http://127.18.63.65:38481/ using document root <none> and password file <none>
I20260808 07:07:15.609570 31094 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:15.609645 31094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:15.609807 31094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:15.610855 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "b2b1a407d24f40739d1a88263a1dac06"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6a8c1418a10f04450356eaa0b9d1b013"
server_key_iv: "40fdc520aa1d5ef349cf9009c0099364"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.611222 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "b2b1a407d24f40739d1a88263a1dac06"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6a8c1418a10f04450356eaa0b9d1b013"
server_key_iv: "40fdc520aa1d5ef349cf9009c0099364"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.612530 31094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:07:15.613240 31110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.613440 31094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:07:15.613545 31094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "b2b1a407d24f40739d1a88263a1dac06"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "6a8c1418a10f04450356eaa0b9d1b013"
server_key_iv: "40fdc520aa1d5ef349cf9009c0099364"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.613631 31094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:15.632840 31094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:15.633173 31094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:15.633395 31094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:15.633649 31094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:15.633996 31094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:15.634055 31094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.634099 31094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:15.634124 31094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.640326 31094 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:44549
I20260808 07:07:15.640388 31251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:44549 every 8 connection(s)
I20260808 07:07:15.640772 31094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:07:15.643289 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31094
I20260808 07:07:15.643390 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:07:15.643590 18685 external_mini_cluster.cc:1428] Setting key 40a63e328b252e6f297cc08a93fb9a39
I20260808 07:07:15.644676 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--builtin_ntp_servers=127.18.63.84:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:07:15.646922 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.641562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:37337 (local address 127.18.63.126:44677)
0808 07:07:15.641636 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.641638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.641915 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:07:15.642436 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.642438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.642448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.642497 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.643314 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.643903 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.645332 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.645538 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.645596 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.646563 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:15.646564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:15.646565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:15.646570 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:15.646590 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:15.646595 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:15.646659 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:15.646807 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 07:07:15.646836 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:07:15.647280 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.641540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44677 (local address 127.18.63.65:37337)
0808 07:07:15.641777 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:15.641810 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:15.642345 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:15.642558 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:15.642561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:15.642569 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:15.643204 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.643212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.643994 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.643995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.645247 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.645250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.645633 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.645634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.645796 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.646351 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:15.646351 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:15.646395 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:15.646425 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:15.646692 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:15.646698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:15.646701 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:07:15.647026 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 07:07:15.647130 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260808 07:07:15.647755 31252 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44677
I20260808 07:07:15.647846 31252 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:15.648033 31252 heartbeater.cc:511] Master 127.18.63.126:44677 requested a full tablet report, sending...
I20260808 07:07:15.648561 31014 ts_manager.cc:194] Registered new tserver with Master: b2b1a407d24f40739d1a88263a1dac06 (127.18.63.65:44549)
I20260808 07:07:15.649206 31014 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.65:37337
W20260808 07:07:15.730461 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:15.730715 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:15.730749 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:15.731881 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:15.731949 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:15.731993 31255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:07:15.733330 31255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:15.733660 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:15.733973 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:15.739292 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.739322 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.739317 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:15.739574 31255 server_base.cc:1034] running on GCE node
I20260808 07:07:15.739820 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:15.740062 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:15.741227 31255 hybrid_clock.cc:648] HybridClock initialized: now 1786172835741217 us; error 29 us; skew 500 ppm
I20260808 07:07:15.742602 31255 webserver.cc:460] Webserver started at http://127.18.63.66:38905/ using document root <none> and password file <none>
I20260808 07:07:15.742844 31255 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:15.742913 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:15.743070 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:15.744244 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "fed54b5879874674a315833e81a759fc"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "853687134f412f5d109ab0d108849141"
server_key_iv: "8ff4541425377fd79f3ac87cf5c8b95f"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.744630 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "fed54b5879874674a315833e81a759fc"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "853687134f412f5d109ab0d108849141"
server_key_iv: "8ff4541425377fd79f3ac87cf5c8b95f"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.746037 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:07:15.746778 31271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.747002 31255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:15.747097 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "fed54b5879874674a315833e81a759fc"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "853687134f412f5d109ab0d108849141"
server_key_iv: "8ff4541425377fd79f3ac87cf5c8b95f"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.747155 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:15.759450 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:15.759709 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:15.759811 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:15.759999 31255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:15.760260 31255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:15.760290 31255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.760311 31255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:15.760324 31255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.766330 31255 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:36355
I20260808 07:07:15.766408 31414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:36355 every 8 connection(s)
I20260808 07:07:15.766734 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:07:15.771507 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31255
I20260808 07:07:15.771622 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:07:15.771844 18685 external_mini_cluster.cc:1428] Setting key af1cad39656b05773ab09afb22aebb6b
I20260808 07:07:15.771934 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.767504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:39973 (local address 127.18.63.126:44677)
0808 07:07:15.767573 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.767574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.767830 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:07:15.768324 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.768324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.768334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.768395 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.769054 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.769570 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.770538 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.770725 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.770775 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.771592 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:15.771594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:15.771595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:15.771601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:15.771621 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:15.771638 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:15.771698 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:15.771817 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 07:07:15.771844 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:07:15.772146 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.767466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44677 (local address 127.18.63.66:39973)
0808 07:07:15.767697 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:15.767739 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:15.768241 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:15.768466 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:15.768470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:15.768478 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:15.768982 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.768986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.769653 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.769654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.770459 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.770462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.770799 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.770799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.770907 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.771451 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:15.771451 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:15.771479 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:15.771502 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:15.771718 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:15.771724 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:15.771727 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:07:15.771795 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:07:15.771879 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260808 07:07:15.772599 31415 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44677
I20260808 07:07:15.772704 31415 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:15.772886 31415 heartbeater.cc:511] Master 127.18.63.126:44677 requested a full tablet report, sending...
I20260808 07:07:15.772921 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--builtin_ntp_servers=127.18.63.84:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 07:07:15.773324 31013 ts_manager.cc:194] Registered new tserver with Master: fed54b5879874674a315833e81a759fc (127.18.63.66:36355)
I20260808 07:07:15.773758 31013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.66:39973
W20260808 07:07:15.857689 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:15.857895 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:15.857916 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:15.859062 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:15.859102 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:15.859164 31419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:07:15.860498 31419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:15.860718 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:15.861007 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:15.866194 31426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.866257 31429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:15.866269 31424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:15.866429 31419 server_base.cc:1034] running on GCE node
I20260808 07:07:15.866673 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:15.866930 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:15.868088 31419 hybrid_clock.cc:648] HybridClock initialized: now 1786172835868077 us; error 42 us; skew 500 ppm
I20260808 07:07:15.869457 31419 webserver.cc:460] Webserver started at http://127.18.63.67:35115/ using document root <none> and password file <none>
I20260808 07:07:15.869724 31419 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:15.869771 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:15.869863 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:15.870990 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "e33d238efb0842bcb2c0e352034520e1"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "b52d17a629391011127a29b2f97298ef"
server_key_iv: "dc0949028b0afb34700011ac94f45661"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.871337 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "e33d238efb0842bcb2c0e352034520e1"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "b52d17a629391011127a29b2f97298ef"
server_key_iv: "dc0949028b0afb34700011ac94f45661"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.872581 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:07:15.873262 31437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.873466 31419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:07:15.873564 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "e33d238efb0842bcb2c0e352034520e1"
format_stamp: "Formatted at 2026-08-08 07:07:15 on dist-test-slave-bw88"
server_key: "b52d17a629391011127a29b2f97298ef"
server_key_iv: "dc0949028b0afb34700011ac94f45661"
server_key_version: "encryptionkey@0"
I20260808 07:07:15.873641 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:15.883701 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:15.883987 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:15.884140 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:15.884358 31419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:15.884696 31419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:15.884732 31419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.884753 31419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:15.884800 31419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:15.892491 31419 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:42071
I20260808 07:07:15.892537 31592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:42071 every 8 connection(s)
I20260808 07:07:15.892895 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:07:15.898561 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31419
I20260808 07:07:15.898669 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:07:15.898898 18685 external_mini_cluster.cc:1428] Setting key 9f073d8c03133a3b38500398d358b2c5
I20260808 07:07:15.899722 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.893758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:44011 (local address 127.18.63.126:44677)
0808 07:07:15.893835 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.893838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.894144 (+ 306us) server_negotiation.cc:407] Connection header received
0808 07:07:15.894938 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.894939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.894949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.894995 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.895946 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.896432 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.898064 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.898256 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.898313 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.899375 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:15.899377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:15.899377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:15.899382 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:15.899400 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:15.899405 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:15.899455 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:15.899610 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:07:15.899642 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:07:15.899973 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:15.893726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44677 (local address 127.18.63.67:44011)
0808 07:07:15.893980 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:15.894030 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:15.894827 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:15.895072 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:15.895077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:15.895089 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:15.895852 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.895860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.896526 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.896527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.897969 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:15.897972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:15.898346 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:15.898347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:15.898541 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:15.899219 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:15.899220 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:15.899262 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:15.899293 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:15.899458 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:15.899466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:15.899469 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:07:15.899612 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 07:07:15.899727 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260808 07:07:15.900868 31594 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44677
I20260808 07:07:15.900982 31594 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:15.901208 31594 heartbeater.cc:511] Master 127.18.63.126:44677 requested a full tablet report, sending...
I20260808 07:07:15.901638 31013 ts_manager.cc:194] Registered new tserver with Master: e33d238efb0842bcb2c0e352034520e1 (127.18.63.67:42071)
I20260808 07:07:15.901969 31013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.67:44011
I20260808 07:07:15.912182 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:07:15.914592 31068 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:15.913967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.18.63.126:44677)
0808 07:07:15.914019 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:15.914022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:15.914176 (+ 154us) server_negotiation.cc:407] Connection header received
0808 07:07:15.914285 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:15.914285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:15.914295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:15.914338 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:15.914498 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40870: BlockingRecv error: recv got EOF from 127.0.0.1:40870 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:07:15.914635 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31094
I20260808 07:07:15.946587 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31255
I20260808 07:07:15.969859 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31419
I20260808 07:07:15.978688 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 30978
2026-08-08T07:07:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (698 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T07:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:07:16Z Disabled control of system clock
I20260808 07:07:16.013816 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:40349
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:40349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:07:16.101645 31608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:16.101886 31608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:16.101922 31608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:16.103052 31608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:07:16.103121 31608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:16.103154 31608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:16.103183 31608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:07:16.103209 31608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:07:16.104559 31608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:40349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:40349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:16.104848 31608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:16.105139 31608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:16.110127 31617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.110152 31616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:16.110204 31608 server_base.cc:1034] running on GCE node
W20260808 07:07:16.110149 31619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:16.110598 31608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:16.110854 31608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:16.111990 31608 hybrid_clock.cc:648] HybridClock initialized: now 1786172836111979 us; error 29 us; skew 500 ppm
I20260808 07:07:16.113189 31608 webserver.cc:460] Webserver started at http://127.18.63.126:46611/ using document root <none> and password file <none>
I20260808 07:07:16.113471 31608 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:16.113519 31608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:16.113690 31608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:16.114814 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "4e41436cb1c04b6b971f27cbef786f8f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "8354f1d510c8628ff1643c9b1d5f4f7b"
server_key_iv: "0301d36f1f5df4c0e28032bf3916ca7a"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.115186 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "4e41436cb1c04b6b971f27cbef786f8f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "8354f1d510c8628ff1643c9b1d5f4f7b"
server_key_iv: "0301d36f1f5df4c0e28032bf3916ca7a"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.116477 31608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:07:16.117172 31627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.117368 31608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:07:16.117484 31608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "4e41436cb1c04b6b971f27cbef786f8f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "8354f1d510c8628ff1643c9b1d5f4f7b"
server_key_iv: "0301d36f1f5df4c0e28032bf3916ca7a"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.117566 31608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:16.120796 31608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:16.121058 31608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:16.121217 31608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:16.125015 31608 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:40349
I20260808 07:07:16.125048 31696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:40349 every 8 connection(s)
I20260808 07:07:16.125429 31608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:07:16.126148 31698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:16.128309 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31608
I20260808 07:07:16.128434 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:07:16.128618 18685 external_mini_cluster.cc:1428] Setting key a97edbff3ae248a5db4e16b137756551
I20260808 07:07:16.129001 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f: Bootstrap starting.
I20260808 07:07:16.129716 31698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:16.130016 31698 log.cc:824] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:16.130972 31698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f: No bootstrap required, opened a new log
I20260808 07:07:16.131297 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.129173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.18.63.126:40349)
0808 07:07:16.129415 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:16.129422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:16.129447 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:07:16.129932 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:16.129939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:16.129980 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:16.130098 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:16.130821 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:16.130825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:16.130826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:16.130830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:16.130878 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:16.130892 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:16.130988 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:16.131021 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 07:07:16.131079 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260808 07:07:16.133061 31698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } }
I20260808 07:07:16.133193 31698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:16.133214 31698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e41436cb1c04b6b971f27cbef786f8f, State: Initialized, Role: FOLLOWER
I20260808 07:07:16.133293 31698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } }
I20260808 07:07:16.133417 31698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:07:16.133443 31698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:07:16.133477 31698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:16.133980 31698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } }
I20260808 07:07:16.134064 31698 leader_election.cc:302] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e41436cb1c04b6b971f27cbef786f8f; no voters:
I20260808 07:07:16.134188 31698 leader_election.cc:288] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:07:16.134270 31703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:07:16.134425 31703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [term 1 LEADER]: Becoming Leader. State: Replica: 4e41436cb1c04b6b971f27cbef786f8f, State: Running, Role: LEADER
I20260808 07:07:16.134474 31698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:07:16.134526 31703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } }
I20260808 07:07:16.134893 31704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e41436cb1c04b6b971f27cbef786f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } } }
I20260808 07:07:16.135017 31704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:16.134898 31706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e41436cb1c04b6b971f27cbef786f8f. Latest consensus state: current_term: 1 leader_uuid: "4e41436cb1c04b6b971f27cbef786f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e41436cb1c04b6b971f27cbef786f8f" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 40349 } } }
I20260808 07:07:16.135098 31706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:16.135254 31710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:07:16.135771 31710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:07:16.137302 31710 catalog_manager.cc:1347] Generated new cluster ID: da924eed36a1409791f4d559f6c6b4d8
I20260808 07:07:16.137374 31710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:07:16.144982 31710 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:07:16.145146 31710 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:07:16.149410 31710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e41436cb1c04b6b971f27cbef786f8f: Generated new TSK 0
I20260808 07:07:16.149581 31710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:07:16.154968 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--builtin_ntp_servers=127.18.63.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:07:16.243248 31730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:16.243515 31730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:16.243561 31730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:16.244751 31730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:16.244814 31730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:16.244855 31730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:07:16.246304 31730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:16.246593 31730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:16.246917 31730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:16.253165 31736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.253144 31739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.253144 31737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:16.253471 31730 server_base.cc:1034] running on GCE node
I20260808 07:07:16.253716 31730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:16.253993 31730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:16.255198 31730 hybrid_clock.cc:648] HybridClock initialized: now 1786172836255171 us; error 48 us; skew 500 ppm
I20260808 07:07:16.256568 31730 webserver.cc:460] Webserver started at http://127.18.63.65:38667/ using document root <none> and password file <none>
I20260808 07:07:16.256839 31730 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:16.256913 31730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:16.257063 31730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:16.258241 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "016b9926542b4f08b925be4d7c745a9f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "5710aac8152a9e1c3659690efb127a0a"
server_key_iv: "dfda325b6dadf2037d8e103345d112de"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.258626 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "016b9926542b4f08b925be4d7c745a9f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "5710aac8152a9e1c3659690efb127a0a"
server_key_iv: "dfda325b6dadf2037d8e103345d112de"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.259968 31730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:07:16.260780 31746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.260978 31730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:07:16.261089 31730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "016b9926542b4f08b925be4d7c745a9f"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "5710aac8152a9e1c3659690efb127a0a"
server_key_iv: "dfda325b6dadf2037d8e103345d112de"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.261186 31730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:16.264642 31730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:16.264914 31730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:16.265060 31730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:16.265281 31730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:16.265646 31730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:16.265709 31730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.265750 31730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:16.265779 31730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.271999 31730 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:45041
I20260808 07:07:16.272044 31896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:45041 every 8 connection(s)
I20260808 07:07:16.272356 31730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:07:16.275463 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.273050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48039 (local address 127.18.63.126:40349)
0808 07:07:16.273101 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:16.273103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:16.273381 (+ 278us) server_negotiation.cc:407] Connection header received
0808 07:07:16.273923 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:16.273925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:16.273933 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:16.273984 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:16.275193 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:16.275194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:16.275195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:16.275200 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:16.275215 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:16.275220 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:16.275277 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:16.275377 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:07:16.275393 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:07:16.275678 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.273033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:40349 (local address 127.18.63.65:48039)
0808 07:07:16.273240 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:16.273272 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:16.273806 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:16.274052 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:16.274056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:16.274066 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:16.275036 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:16.275036 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:16.275063 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:16.275091 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:16.275283 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:16.275289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:16.275292 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:07:16.275390 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:07:16.275450 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260808 07:07:16.276247 31898 heartbeater.cc:347] Connected to a master server at 127.18.63.126:40349
I20260808 07:07:16.276342 31898 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:16.276507 31898 heartbeater.cc:511] Master 127.18.63.126:40349 requested a full tablet report, sending...
I20260808 07:07:16.277031 31649 ts_manager.cc:194] Registered new tserver with Master: 016b9926542b4f08b925be4d7c745a9f (127.18.63.65:45041)
I20260808 07:07:16.280859 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31730
I20260808 07:07:16.280941 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:07:16.281141 18685 external_mini_cluster.cc:1428] Setting key 7d3a80e23f00b4361c734324d1385020
I20260808 07:07:16.282145 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--builtin_ntp_servers=127.18.63.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:07:16.366380 31901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:16.366619 31901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:16.366659 31901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:16.367820 31901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:16.367882 31901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:16.367925 31901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:07:16.369326 31901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:16.369620 31901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:16.369922 31901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:16.375094 31909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.375128 31908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.375104 31911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:16.375484 31901 server_base.cc:1034] running on GCE node
I20260808 07:07:16.375684 31901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:16.375877 31901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:16.377020 31901 hybrid_clock.cc:648] HybridClock initialized: now 1786172836377011 us; error 32 us; skew 500 ppm
I20260808 07:07:16.378391 31901 webserver.cc:460] Webserver started at http://127.18.63.66:46767/ using document root <none> and password file <none>
I20260808 07:07:16.378713 31901 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:16.378772 31901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:16.378961 31901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:16.380057 31901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "3100255ee0ea4d438f216090d4043068"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "1e0d1626810a97e6ac30f497399ac0aa"
server_key_iv: "22fefe755d157016f36649ebe1443823"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.380410 31901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "3100255ee0ea4d438f216090d4043068"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "1e0d1626810a97e6ac30f497399ac0aa"
server_key_iv: "22fefe755d157016f36649ebe1443823"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.381848 31901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:07:16.382618 31921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.382803 31901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:07:16.382913 31901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "3100255ee0ea4d438f216090d4043068"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "1e0d1626810a97e6ac30f497399ac0aa"
server_key_iv: "22fefe755d157016f36649ebe1443823"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.382990 31901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:16.386512 31901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:16.386770 31901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:16.386916 31901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:16.387130 31901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:16.387487 31901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:16.387544 31901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.387584 31901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:16.387611 31901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.393522 31901 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:45185
I20260808 07:07:16.393576 32061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:45185 every 8 connection(s)
I20260808 07:07:16.393901 31901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:07:16.396814 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.394664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:34543 (local address 127.18.63.126:40349)
0808 07:07:16.394738 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:16.394741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:16.394961 (+ 220us) server_negotiation.cc:407] Connection header received
0808 07:07:16.395403 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:16.395404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:16.395409 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:16.395464 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:16.396634 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:16.396636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:16.396636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:16.396639 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:16.396653 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:16.396657 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:16.396701 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:16.396747 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:07:16.396760 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:07:16.397033 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.394650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:40349 (local address 127.18.63.66:34543)
0808 07:07:16.394831 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:16.394864 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:16.395320 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:16.395559 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:16.395562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:16.395570 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:16.396484 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:16.396485 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:16.396513 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:16.396542 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:16.396700 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:16.396703 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:16.396705 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:07:16.396754 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:07:16.396799 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260808 07:07:16.397588 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 31901
I20260808 07:07:16.397569 32062 heartbeater.cc:347] Connected to a master server at 127.18.63.126:40349
I20260808 07:07:16.397686 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:07:16.397688 32062 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:16.397850 32062 heartbeater.cc:511] Master 127.18.63.126:40349 requested a full tablet report, sending...
I20260808 07:07:16.397917 18685 external_mini_cluster.cc:1428] Setting key 34273c0cab20bdcc861adebd13b0ea80
I20260808 07:07:16.398263 31649 ts_manager.cc:194] Registered new tserver with Master: 3100255ee0ea4d438f216090d4043068 (127.18.63.66:45185)
I20260808 07:07:16.399083 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--builtin_ntp_servers=127.18.63.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 07:07:16.483894 32065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:16.484091 32065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:16.484117 32065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:16.485292 32065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:16.485351 32065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:16.485415 32065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:07:16.486869 32065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:40349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:16.487049 32065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:16.487284 32065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:16.492399 32074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.492462 32076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:16.492416 32073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:16.492735 32065 server_base.cc:1034] running on GCE node
I20260808 07:07:16.492954 32065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:16.493227 32065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:16.494397 32065 hybrid_clock.cc:648] HybridClock initialized: now 1786172836494376 us; error 33 us; skew 500 ppm
I20260808 07:07:16.495749 32065 webserver.cc:460] Webserver started at http://127.18.63.67:37143/ using document root <none> and password file <none>
I20260808 07:07:16.495994 32065 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:16.496066 32065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:16.496212 32065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:16.497280 32065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "d3b3a1461b784ec9aedeb4f1774365d6"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "532a1df073d9ee3d98defe97be1d40d5"
server_key_iv: "75cf9900bce122498a6b5d1119dd6c78"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.497728 32065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "d3b3a1461b784ec9aedeb4f1774365d6"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "532a1df073d9ee3d98defe97be1d40d5"
server_key_iv: "75cf9900bce122498a6b5d1119dd6c78"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.499055 32065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:16.499751 32084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.499927 32065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:07:16.500031 32065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "d3b3a1461b784ec9aedeb4f1774365d6"
format_stamp: "Formatted at 2026-08-08 07:07:16 on dist-test-slave-bw88"
server_key: "532a1df073d9ee3d98defe97be1d40d5"
server_key_iv: "75cf9900bce122498a6b5d1119dd6c78"
server_key_version: "encryptionkey@0"
I20260808 07:07:16.500111 32065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:16.503492 32065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:16.503762 32065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:16.503904 32065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:16.504115 32065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:16.504422 32065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:16.504477 32065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.504515 32065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:16.504542 32065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:16.510622 32065 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:42077
I20260808 07:07:16.510661 32223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:42077 every 8 connection(s)
I20260808 07:07:16.510974 32065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:07:16.514032 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.511646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:51269 (local address 127.18.63.126:40349)
0808 07:07:16.511694 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:16.511696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:16.511959 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:07:16.512469 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:16.512470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:16.512478 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:16.512532 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:16.513739 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:16.513741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:16.513741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:07:16.513746 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:16.513763 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:07:16.513767 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:16.513840 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:16.513934 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 07:07:16.513961 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:07:16.514259 32226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:16.511651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:40349 (local address 127.18.63.67:51269)
0808 07:07:16.511843 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:16.511874 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:16.512355 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:16.512613 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:16.512617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:16.512628 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:16.513587 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:07:16.513587 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:16.513615 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:07:16.513645 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:16.513838 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:16.513845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:16.513848 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:07:16.513967 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 07:07:16.514028 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260808 07:07:16.514797 32224 heartbeater.cc:347] Connected to a master server at 127.18.63.126:40349
I20260808 07:07:16.514900 32224 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:16.514930 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 32065
I20260808 07:07:16.515023 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:07:16.515062 32224 heartbeater.cc:511] Master 127.18.63.126:40349 requested a full tablet report, sending...
I20260808 07:07:16.515254 18685 external_mini_cluster.cc:1428] Setting key 790037da59f3c417b2f4d4bd94376aff
I20260808 07:07:16.515486 31648 ts_manager.cc:194] Registered new tserver with Master: d3b3a1461b784ec9aedeb4f1774365d6 (127.18.63.67:42077)
I20260808 07:07:16.516139 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 07:07:16.517936 31701 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:07:16.517475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.18.63.126:40349)
0808 07:07:16.517517 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:16.517520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:16.517683 (+ 163us) server_negotiation.cc:407] Connection header received
0808 07:07:16.517718 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:16.517719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:16.517723 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:16.517767 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:16.517879 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33302: BlockingRecv error: recv got EOF from 127.0.0.1:33302 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:07:16.518220 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31730
I20260808 07:07:16.537449 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31901
I20260808 07:07:16.549136 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 32065
I20260808 07:07:16.560148 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 31608
2026-08-08T07:07:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (579 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:07:16 dist-test-slave-bw88 krb5kdc[32235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:07:16 dist-test-slave-bw88 krb5kdc[32235](info): set up 2 sockets
Aug 08 07:07:16 dist-test-slave-bw88 krb5kdc[32235](info): commencing operation
krb5kdc: starting...
W20260808 07:07:17.749488 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:07:17 dist-test-slave-bw88 krb5kdc[32235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:07:17.872972 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33497
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:07:17.958670 32252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:17.958932 32252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:17.958976 32252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:17.960135 32252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:07:17.960224 32252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:17.960256 32252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:17.960290 32252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:07:17.960319 32252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:07:17.961907 32252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:17.962196 32252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:17.962455 32252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:17.967519 32258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:17.967643 32260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:17.967748 32252 server_base.cc:1034] running on GCE node
W20260808 07:07:17.967510 32257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:17.968031 32252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:17.968341 32252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:17.969532 32252 hybrid_clock.cc:648] HybridClock initialized: now 1786172837969501 us; error 37 us; skew 500 ppm
Aug 08 07:07:17 dist-test-slave-bw88 krb5kdc[32235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172837, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:17.972138 32252 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:07:17.972641 32252 webserver.cc:460] Webserver started at http://127.18.63.126:46053/ using document root <none> and password file <none>
I20260808 07:07:17.972878 32252 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:17.972924 32252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:17.973078 32252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:17.974262 32252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "8cac677c315849b68c0aade5e58b48d5"
format_stamp: "Formatted at 2026-08-08 07:07:17 on dist-test-slave-bw88"
server_key: "3aaefd04ebd86bcd125ce2ad10f19a2b"
server_key_iv: "c84d47ed7765e1815bc3c55510dfabf0"
server_key_version: "encryptionkey@0"
I20260808 07:07:17.974637 32252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "8cac677c315849b68c0aade5e58b48d5"
format_stamp: "Formatted at 2026-08-08 07:07:17 on dist-test-slave-bw88"
server_key: "3aaefd04ebd86bcd125ce2ad10f19a2b"
server_key_iv: "c84d47ed7765e1815bc3c55510dfabf0"
server_key_version: "encryptionkey@0"
I20260808 07:07:17.975991 32252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:07:17.976747 32269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:17.976928 32252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:07:17.977023 32252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8cac677c315849b68c0aade5e58b48d5"
format_stamp: "Formatted at 2026-08-08 07:07:17 on dist-test-slave-bw88"
server_key: "3aaefd04ebd86bcd125ce2ad10f19a2b"
server_key_iv: "c84d47ed7765e1815bc3c55510dfabf0"
server_key_version: "encryptionkey@0"
I20260808 07:07:17.977097 32252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:18.005741 32252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:18.007727 32252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:18.007894 32252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:18.013145 32252 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33497
I20260808 07:07:18.013183 32336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33497 every 8 connection(s)
I20260808 07:07:18.013595 32252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:07:18.014635 32337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:18.016952 32337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: Bootstrap starting.
I20260808 07:07:18.017571 32337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:18.017833 32337 log.cc:824] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:18.018275 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 32252
I20260808 07:07:18.018356 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:07:18.018579 18685 external_mini_cluster.cc:1428] Setting key 1084d72ec1f241e73876c8873adbb001
I20260808 07:07:18.018790 32337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: No bootstrap required, opened a new log
I20260808 07:07:18.020639 32337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } }
I20260808 07:07:18.020805 32337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:18.020874 32337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cac677c315849b68c0aade5e58b48d5, State: Initialized, Role: FOLLOWER
I20260808 07:07:18.021013 32337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } }
I20260808 07:07:18.021126 32337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:07:18.021183 32337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:07:18.021234 32337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:18.022091 32337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } }
I20260808 07:07:18.022224 32337 leader_election.cc:302] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cac677c315849b68c0aade5e58b48d5; no voters:
I20260808 07:07:18.022385 32337 leader_election.cc:288] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:07:18.022409 32345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:07:18.022536 32345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [term 1 LEADER]: Becoming Leader. State: Replica: 8cac677c315849b68c0aade5e58b48d5, State: Running, Role: LEADER
I20260808 07:07:18.022655 32337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:07:18.022725 32345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } }
I20260808 07:07:18.023180 32345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cac677c315849b68c0aade5e58b48d5. Latest consensus state: current_term: 1 leader_uuid: "8cac677c315849b68c0aade5e58b48d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } } }
I20260808 07:07:18.023346 32345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:18.023850 32355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:07:18.024150 32347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cac677c315849b68c0aade5e58b48d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac677c315849b68c0aade5e58b48d5" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33497 } } }
I20260808 07:07:18.024297 32347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5 [sys.catalog]: This master's current role is: LEADER
I20260808 07:07:18.024433 32355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:07:18.026057 32355 catalog_manager.cc:1347] Generated new cluster ID: 3cbf33a4896e4298b69e238f28091425
I20260808 07:07:18.026120 32355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:07:18.031469 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.019203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.18.63.126:33497)
0808 07:07:18.019396 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.019404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.019428 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:07:18.020647 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.020655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.020706 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.020861 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:18.021683 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.022819 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.023845 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.024478 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.024610 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.027730 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:18.027734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:18.027734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:18.027745 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:18.029821 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.030234 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.030237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.030238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.030358 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.030586 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.030587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.030587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.030871 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:18.031024 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.031145 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.031275 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260808 07:07:18.034849 32355 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:07:18.035377 32355 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:07:18.037844 32355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: Generated new TSK 0
I20260808 07:07:18.038023 32355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:07:18.066540 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:07:18.151412 32369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:18.151661 32369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:18.151703 32369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:18.152853 32369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:18.152920 32369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:18.152966 32369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:07:18.154631 32369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:18.154901 32369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:18.155151 32369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:18.160492 32377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:18.160537 32376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:18.160602 32369 server_base.cc:1034] running on GCE node
W20260808 07:07:18.160645 32379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:18.160940 32369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:18.161221 32369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:18.162389 32369 hybrid_clock.cc:648] HybridClock initialized: now 1786172838162381 us; error 38 us; skew 500 ppm
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:18.165055 32369 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:07:18.165576 32369 webserver.cc:460] Webserver started at http://127.18.63.65:36039/ using document root <none> and password file <none>
I20260808 07:07:18.165812 32369 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:18.165884 32369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:18.166049 32369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:18.167111 32369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "f84a23017e6548538d344c6eecc56502"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "accb0da8ded0b650900121a4c2f433e5"
server_key_iv: "6f14102190845153a316a06c28909f3a"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.167496 32369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "f84a23017e6548538d344c6eecc56502"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "accb0da8ded0b650900121a4c2f433e5"
server_key_iv: "6f14102190845153a316a06c28909f3a"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.168849 32369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:07:18.169654 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.169836 32369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:18.169950 32369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "f84a23017e6548538d344c6eecc56502"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "accb0da8ded0b650900121a4c2f433e5"
server_key_iv: "6f14102190845153a316a06c28909f3a"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.170028 32369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:18.178701 32369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:18.180718 32369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:18.180894 32369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:18.181134 32369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:18.181533 32369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:18.181589 32369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.181636 32369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:18.181665 32369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.188709 32369 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:41437
I20260808 07:07:18.188750 32538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:41437 every 8 connection(s)
I20260808 07:07:18.189106 32369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:07:18.192721 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 32369
I20260808 07:07:18.192840 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:07:18.193056 18685 external_mini_cluster.cc:1428] Setting key 86e12782f4fa9c7aba2b0b8ee8de19cf
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:07:18.199234 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.189904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:53201 (local address 127.18.63.126:33497)
0808 07:07:18.189989 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.189991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.190337 (+ 346us) server_negotiation.cc:407] Connection header received
0808 07:07:18.190732 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.190734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.190748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.190809 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:18.191704 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.192212 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.193402 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.193645 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.193727 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.196185 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:18.196187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:18.196188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:18.196198 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:18.197742 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.198180 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.198181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.198181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.198244 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.198521 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.198523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.198523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.198652 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:18.198729 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.199027 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.199139 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 07:07:18.199430 32541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.189888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33497 (local address 127.18.63.65:53201)
0808 07:07:18.190070 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.190111 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.190637 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.190861 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.190864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.191428 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:18.191617 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.191622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.192297 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.192298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.193299 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.193303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.193734 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.193735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.193913 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.194457 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:18.194465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:18.196074 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:18.197846 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.197854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.197856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.198094 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.198320 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.198321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.198321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.198433 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.198733 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:18.198741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:18.198887 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.198989 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.199176 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260808 07:07:18.200151 32539 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33497
I20260808 07:07:18.200235 32539 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:18.200433 32539 heartbeater.cc:511] Master 127.18.63.126:33497 requested a full tablet report, sending...
I20260808 07:07:18.201018 32290 ts_manager.cc:194] Registered new tserver with Master: f84a23017e6548538d344c6eecc56502 (127.18.63.65:41437)
I20260808 07:07:18.202008 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:53201
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:07:18.220721 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:07:18.306694 32544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:18.306898 32544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:18.306923 32544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:18.308032 32544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:18.308066 32544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:18.308091 32544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:07:18.309602 32544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:18.309783 32544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:18.309993 32544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:18.315155 32554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:18.315248 32550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:18.315260 32544 server_base.cc:1034] running on GCE node
W20260808 07:07:18.315155 32549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:18.315644 32544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:18.315902 32544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:18.317077 32544 hybrid_clock.cc:648] HybridClock initialized: now 1786172838317073 us; error 36 us; skew 500 ppm
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:18.319715 32544 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:07:18.320251 32544 webserver.cc:460] Webserver started at http://127.18.63.66:33385/ using document root <none> and password file <none>
I20260808 07:07:18.320501 32544 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:18.320574 32544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:18.320744 32544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:18.321941 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "247907a7adb843e3b9a05befbbfd55ee"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "cd74da5109e916d2f6b6759f062313d6"
server_key_iv: "bb7c407950eb41b6cd44e0ac8a9444a5"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.322368 32544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "247907a7adb843e3b9a05befbbfd55ee"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "cd74da5109e916d2f6b6759f062313d6"
server_key_iv: "bb7c407950eb41b6cd44e0ac8a9444a5"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.323997 32544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:07:18.324965 32561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.325165 32544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:07:18.325282 32544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "247907a7adb843e3b9a05befbbfd55ee"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "cd74da5109e916d2f6b6759f062313d6"
server_key_iv: "bb7c407950eb41b6cd44e0ac8a9444a5"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.325397 32544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:18.337289 32544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:18.339066 32544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:18.339243 32544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:18.339473 32544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:18.339795 32544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:18.339854 32544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.339893 32544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:18.339921 32544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.346119 32544 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:45711
I20260808 07:07:18.346167 32699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:45711 every 8 connection(s)
I20260808 07:07:18.346477 32544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:07:18.355577 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.347102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:49379 (local address 127.18.63.126:33497)
0808 07:07:18.347177 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.347179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.347361 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:07:18.347852 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.347853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.347862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.347920 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:18.348832 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.349333 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.350408 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.350628 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.350698 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.352865 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:18.352866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:18.352867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:18.352874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:18.354315 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.354689 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.354690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.354690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.354743 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.355030 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.355031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.355032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.355153 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:18.355208 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.355440 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.355511 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:07:18.355726 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.347059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33497 (local address 127.18.63.66:49379)
0808 07:07:18.347226 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.347267 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.347768 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.348009 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.348013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.348568 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:18.348757 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.348761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.349499 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.349500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.350309 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.350312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.350736 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.350737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.350859 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.351333 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:18.351342 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:18.352779 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:18.354390 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.354396 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.354398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.354619 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.354817 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.354818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.354818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.354980 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.355214 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:18.355219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:18.355330 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.355410 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.355529 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 07:07:18.356297 32701 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33497
I20260808 07:07:18.356391 32701 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:18.356432 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 32544
I20260808 07:07:18.356555 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:07:18.356639 32701 heartbeater.cc:511] Master 127.18.63.126:33497 requested a full tablet report, sending...
I20260808 07:07:18.356792 18685 external_mini_cluster.cc:1428] Setting key e75ef07b23c33cf8dc9c5fb52c0939fc
I20260808 07:07:18.357091 32290 ts_manager.cc:194] Registered new tserver with Master: 247907a7adb843e3b9a05befbbfd55ee (127.18.63.66:45711)
I20260808 07:07:18.357707 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:49379
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:07:18.382366 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:07:18.466491 32708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:07:18.466740 32708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:07:18.466780 32708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:07:18.467911 32708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:07:18.467983 32708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:07:18.468029 32708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:07:18.469633 32708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:07:18.469913 32708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:07:18.470161 32708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:07:18.475327 32713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:18.475306 32716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:07:18.475306 32714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:07:18.475715 32708 server_base.cc:1034] running on GCE node
I20260808 07:07:18.475983 32708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:07:18.476199 32708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:07:18.477500 32708 hybrid_clock.cc:648] HybridClock initialized: now 1786172838477481 us; error 27 us; skew 500 ppm
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:07:18.480381 32708 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:07:18.480914 32708 webserver.cc:460] Webserver started at http://127.18.63.67:45271/ using document root <none> and password file <none>
I20260808 07:07:18.481204 32708 fs_manager.cc:359] Metadata directory not provided
I20260808 07:07:18.481285 32708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:07:18.481498 32708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:07:18.482645 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "e7045dfbebb045f0b7aac6556e997306"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "da9c959d420dd22512087b8908b9e65a"
server_key_iv: "686557edc0cdb16edc308df4ac23eadd"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.483068 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "e7045dfbebb045f0b7aac6556e997306"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "da9c959d420dd22512087b8908b9e65a"
server_key_iv: "686557edc0cdb16edc308df4ac23eadd"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.484601 32708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:07:18.485440 32724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.485644 32708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:07:18.485760 32708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "e7045dfbebb045f0b7aac6556e997306"
format_stamp: "Formatted at 2026-08-08 07:07:18 on dist-test-slave-bw88"
server_key: "da9c959d420dd22512087b8908b9e65a"
server_key_iv: "686557edc0cdb16edc308df4ac23eadd"
server_key_version: "encryptionkey@0"
I20260808 07:07:18.485857 32708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:07:18.503681 32708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:07:18.505918 32708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:07:18.506119 32708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:07:18.506397 32708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:07:18.506778 32708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:07:18.506850 32708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.506902 32708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:07:18.506929 32708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:07:18.513708 32708 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:33701
I20260808 07:07:18.513763 399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:33701 every 8 connection(s)
I20260808 07:07:18.514118 32708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:07:18.517407 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 32708
I20260808 07:07:18.517524 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:07:18.517760 18685 external_mini_cluster.cc:1428] Setting key f0b6bfb76827f80f382251a32293cc70
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[32235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172838, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:07:18.524291 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.514802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:38559 (local address 127.18.63.126:33497)
0808 07:07:18.514890 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.514892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.515096 (+ 204us) server_negotiation.cc:407] Connection header received
0808 07:07:18.515665 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.515666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.515680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.515744 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:18.516756 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.517266 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.518748 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.519088 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.519154 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.521429 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:18.521431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:18.521431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:18.521440 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:18.522848 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.523225 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.523226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.523227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.523286 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.523569 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.523570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.523571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.523686 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:18.523745 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.524037 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.524106 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:07:18.524478 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.514756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33497 (local address 127.18.63.67:38559)
0808 07:07:18.514951 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.514989 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.515526 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.515822 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.515825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.516421 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:07:18.516652 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.516658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.517380 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.517382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.518665 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.518668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.519171 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.519172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.519278 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.519808 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:07:18.519818 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:07:18.521283 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:07:18.522926 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.522932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.522934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.523156 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.523392 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:07:18.523392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:07:18.523392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:07:18.523492 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:07:18.523747 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:07:18.523753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:07:18.523923 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.524004 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.524183 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
I20260808 07:07:18.525022 400 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33497
I20260808 07:07:18.525106 400 heartbeater.cc:464] Registering TS with master...
I20260808 07:07:18.525305 400 heartbeater.cc:511] Master 127.18.63.126:33497 requested a full tablet report, sending...
I20260808 07:07:18.525796 32290 ts_manager.cc:194] Registered new tserver with Master: e7045dfbebb045f0b7aac6556e997306 (127.18.63.67:33701)
I20260808 07:07:18.526180 32290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:38559
I20260808 07:07:18.529915 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:07:18.537689 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.531356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.18.63.126:33497)
0808 07:07:18.531418 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.531420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.531557 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:07:18.531653 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.531654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.531666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.531756 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:07:18.532364 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.532845 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.533713 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.533907 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.534000 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.535256 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:07:18.535258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:07:18.535259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:07:18.535266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:07:18.536441 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.536804 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.536805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.536805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.536867 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:07:18.537073 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:07:18.537074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:07:18.537074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:07:18.537224 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:07:18.537329 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.537495 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.537576 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:07:18.539173 32290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:07:18.540747 32290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:07:18.547624 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44794 (local address 127.18.63.65:41437)
0808 07:07:18.543891 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.543896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.543906 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:18.543950 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.543952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.543969 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.544058 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:07:18.544277 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.544976 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.546540 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547136 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547240 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547310 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.547481 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.547530 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260808 07:07:18.547623 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41437 (local address 127.0.0.1:44794)
0808 07:07:18.543525 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.543534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.543572 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.544144 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.544145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.544147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:07:18.544198 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.544200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.545092 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.545093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.546442 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.546445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547245 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.547339 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547353 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.547460 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.547518 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":38,"thread_start_us":100,"threads_started":1}
I20260808 07:07:18.547798 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.18.63.66:45711)
0808 07:07:18.543926 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.543931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.543941 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:07:18.544060 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.544063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.544079 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.544157 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:07:18.544369 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.545456 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.546512 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547229 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547303 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547357 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.547668 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.547717 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":104,"threads_started":1}
I20260808 07:07:18.547987 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.18.63.67:33701)
0808 07:07:18.543602 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.543607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.543684 (+ 77us) server_negotiation.cc:407] Connection header received
0808 07:07:18.543748 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.543751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.543790 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.543878 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:07:18.544163 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.545301 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.546722 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547427 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547491 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547543 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.547865 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.547919 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260808 07:07:18.547788 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33701 (local address 127.0.0.1:41058)
0808 07:07:18.543579 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.543623 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.543671 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.543982 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.543985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.543987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:07:18.544059 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.544060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.545433 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.545435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.546641 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.546643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547575 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.547635 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547639 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.547691 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.547726 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":98,"threads_started":1}
I20260808 07:07:18.548208 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.543138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:45711 (local address 127.0.0.1:50932)
0808 07:07:18.543311 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.543348 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.543417 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.544239 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.544240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.544242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:07:18.544286 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.544288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.545571 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.545572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.546422 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.546425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.547396 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.547397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.547466 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.547471 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.547520 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.547568 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":59,"threads_started":1}
I20260808 07:07:18.549057 32455 tablet_service.cc:1467] Processing CreateTablet for tablet 29dd8c626bbc4210be45165652ee3a41 (DEFAULT_TABLE table=test-table [id=da927a092bc54dddaaf073f6ad3af889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:07:18.549150 32622 tablet_service.cc:1467] Processing CreateTablet for tablet 29dd8c626bbc4210be45165652ee3a41 (DEFAULT_TABLE table=test-table [id=da927a092bc54dddaaf073f6ad3af889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:07:18.549150 318 tablet_service.cc:1467] Processing CreateTablet for tablet 29dd8c626bbc4210be45165652ee3a41 (DEFAULT_TABLE table=test-table [id=da927a092bc54dddaaf073f6ad3af889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:07:18.549490 32455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dd8c626bbc4210be45165652ee3a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:18.549536 318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dd8c626bbc4210be45165652ee3a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:18.549536 32622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29dd8c626bbc4210be45165652ee3a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:07:18.552127 420 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: Bootstrap starting.
I20260808 07:07:18.552889 421 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Bootstrap starting.
I20260808 07:07:18.553032 420 tablet_bootstrap.cc:654] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:18.553507 422 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Bootstrap starting.
I20260808 07:07:18.553634 421 tablet_bootstrap.cc:654] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:18.553720 420 log.cc:824] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:18.554196 421 log.cc:824] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:18.554252 422 tablet_bootstrap.cc:654] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Neither blocks nor log segments found. Creating new log.
I20260808 07:07:18.554790 422 log.cc:824] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Log is configured to *not* fsync() on all Append() calls
I20260808 07:07:18.554910 420 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: No bootstrap required, opened a new log
I20260808 07:07:18.554991 420 ts_tablet_manager.cc:1397] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:07:18.555208 421 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: No bootstrap required, opened a new log
I20260808 07:07:18.555271 421 ts_tablet_manager.cc:1397] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:07:18.555816 422 tablet_bootstrap.cc:492] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: No bootstrap required, opened a new log
I20260808 07:07:18.555902 422 ts_tablet_manager.cc:1397] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:07:18.557032 420 raft_consensus.cc:348] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.557231 420 raft_consensus.cc:374] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:18.557173 421 raft_consensus.cc:348] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.557269 420 raft_consensus.cc:729] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84a23017e6548538d344c6eecc56502, State: Initialized, Role: FOLLOWER
I20260808 07:07:18.557309 421 raft_consensus.cc:374] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:18.557367 421 raft_consensus.cc:729] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7045dfbebb045f0b7aac6556e997306, State: Initialized, Role: FOLLOWER
I20260808 07:07:18.557427 420 consensus_queue.cc:260] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.557507 421 consensus_queue.cc:260] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.557478 422 raft_consensus.cc:348] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.557675 422 raft_consensus.cc:374] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:07:18.557726 422 raft_consensus.cc:729] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247907a7adb843e3b9a05befbbfd55ee, State: Initialized, Role: FOLLOWER
I20260808 07:07:18.557740 420 ts_tablet_manager.cc:1428] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:07:18.557783 421 ts_tablet_manager.cc:1428] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 07:07:18.557894 400 heartbeater.cc:503] Master 127.18.63.126:33497 was elected leader, sending a full tablet report...
I20260808 07:07:18.557940 32539 heartbeater.cc:503] Master 127.18.63.126:33497 was elected leader, sending a full tablet report...
I20260808 07:07:18.557888 422 consensus_queue.cc:260] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.558197 32701 heartbeater.cc:503] Master 127.18.63.126:33497 was elected leader, sending a full tablet report...
I20260808 07:07:18.558182 422 ts_tablet_manager.cc:1428] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260808 07:07:18.597038 32702 tablet.cc:2369] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:07:18.651140 427 raft_consensus.cc:482] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:07:18.651285 427 raft_consensus.cc:504] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.651754 427 leader_election.cc:288] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 247907a7adb843e3b9a05befbbfd55ee (127.18.63.66:45711), f84a23017e6548538d344c6eecc56502 (127.18.63.65:41437)
I20260808 07:07:18.656035 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.652138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58869 (local address 127.18.63.65:41437)
0808 07:07:18.652230 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.652232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.652477 (+ 245us) server_negotiation.cc:407] Connection header received
0808 07:07:18.652522 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.652523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.652550 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.652607 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:07:18.652762 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.653483 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655047 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.655645 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655726 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.655736 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.655921 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.655962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:07:18.656035 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.652060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:32963 (local address 127.18.63.66:45711)
0808 07:07:18.652174 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.652177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.652190 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:07:18.652237 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.652238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.652248 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.652314 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:07:18.652577 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.653403 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655047 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.655651 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655726 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.655736 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.655921 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.655962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 07:07:18.656036 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.651963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:45711 (local address 127.18.63.67:32963)
0808 07:07:18.652034 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.652049 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.652129 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.652348 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.652349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.652351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:07:18.652493 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.652495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.653570 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.653572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.654941 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.654944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.655741 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.655828 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.655830 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.655894 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.655949 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":101}
I20260808 07:07:18.656046 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.651963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41437 (local address 127.18.63.67:58869)
0808 07:07:18.652394 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.652405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:07:18.652466 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:07:18.652623 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:07:18.652623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:07:18.652624 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:07:18.652693 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.652694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.653570 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.653572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.654941 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:07:18.654944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.655741 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.655742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:07:18.655824 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.655829 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:07:18.655894 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:07:18.655949 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":254,"thread_start_us":149,"threads_started":1}
I20260808 07:07:18.656455 32646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dd8c626bbc4210be45165652ee3a41" candidate_uuid: "e7045dfbebb045f0b7aac6556e997306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247907a7adb843e3b9a05befbbfd55ee" is_pre_election: true
I20260808 07:07:18.656507 32477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dd8c626bbc4210be45165652ee3a41" candidate_uuid: "e7045dfbebb045f0b7aac6556e997306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84a23017e6548538d344c6eecc56502" is_pre_election: true
I20260808 07:07:18.656653 32477 raft_consensus.cc:2454] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7045dfbebb045f0b7aac6556e997306 in term 0.
I20260808 07:07:18.656643 32646 raft_consensus.cc:2454] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7045dfbebb045f0b7aac6556e997306 in term 0.
I20260808 07:07:18.656868 32730 leader_election.cc:302] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7045dfbebb045f0b7aac6556e997306, f84a23017e6548538d344c6eecc56502; no voters:
I20260808 07:07:18.657023 427 raft_consensus.cc:2781] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:07:18.657078 427 raft_consensus.cc:482] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:07:18.657123 427 raft_consensus.cc:3037] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:18.657938 427 raft_consensus.cc:504] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.658097 427 leader_election.cc:288] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [CANDIDATE]: Term 1 election: Requested vote from peers 247907a7adb843e3b9a05befbbfd55ee (127.18.63.66:45711), f84a23017e6548538d344c6eecc56502 (127.18.63.65:41437)
I20260808 07:07:18.658283 32477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dd8c626bbc4210be45165652ee3a41" candidate_uuid: "e7045dfbebb045f0b7aac6556e997306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84a23017e6548538d344c6eecc56502"
I20260808 07:07:18.658277 32646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29dd8c626bbc4210be45165652ee3a41" candidate_uuid: "e7045dfbebb045f0b7aac6556e997306" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247907a7adb843e3b9a05befbbfd55ee"
I20260808 07:07:18.658353 32646 raft_consensus.cc:3037] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:18.658378 32477 raft_consensus.cc:3037] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:07:18.659160 32646 raft_consensus.cc:2454] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7045dfbebb045f0b7aac6556e997306 in term 1.
I20260808 07:07:18.659168 32477 raft_consensus.cc:2454] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7045dfbebb045f0b7aac6556e997306 in term 1.
I20260808 07:07:18.659343 32730 leader_election.cc:302] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7045dfbebb045f0b7aac6556e997306, f84a23017e6548538d344c6eecc56502; no voters:
I20260808 07:07:18.659463 427 raft_consensus.cc:2781] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:07:18.659619 427 raft_consensus.cc:686] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 1 LEADER]: Becoming Leader. State: Replica: e7045dfbebb045f0b7aac6556e997306, State: Running, Role: LEADER
I20260808 07:07:18.659715 427 consensus_queue.cc:237] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } }
I20260808 07:07:18.660607 32289 catalog_manager.cc:5495] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7045dfbebb045f0b7aac6556e997306 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "e7045dfbebb045f0b7aac6556e997306" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7045dfbebb045f0b7aac6556e997306" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 } health_report { overall_health: UNKNOWN } } }
I20260808 07:07:18.669827 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.667481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.18.63.67:33701)
0808 07:07:18.667535 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:07:18.667537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:07:18.667566 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:07:18.667592 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:07:18.667593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:07:18.667609 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:07:18.667662 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:07:18.667770 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.668216 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.669018 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:07:18.669204 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:07:18.669256 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:07:18.669483 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:07:18.669574 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:07:18.669642 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:07:18.669720 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 07:07:18.669737 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:07:18.671686 32477 raft_consensus.cc:1261] T 29dd8c626bbc4210be45165652ee3a41 P f84a23017e6548538d344c6eecc56502 [term 1 FOLLOWER]: Refusing update from remote peer e7045dfbebb045f0b7aac6556e997306: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:07:18.671686 32646 raft_consensus.cc:1261] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 1 FOLLOWER]: Refusing update from remote peer e7045dfbebb045f0b7aac6556e997306: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:07:18.672099 427 consensus_queue.cc:1035] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84a23017e6548538d344c6eecc56502" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:07:18.672282 432 consensus_queue.cc:1035] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [LEADER]: Connected to new peer: Peer: permanent_uuid: "247907a7adb843e3b9a05befbbfd55ee" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:07:18.676266 438 mvcc.cc:205] Tried to move back new op lower bound from 7316163947196424192 to 7316163947150897152. Current Snapshot: MvccSnapshot[applied={T|T < 7316163947196424192 or (T in {7316163947196424192})}]
I20260808 07:07:18.677507 32290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:07:18.677603 32290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:07:18.678371 32290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cac677c315849b68c0aade5e58b48d5: Sending DeleteTablet for 3 replicas of tablet 29dd8c626bbc4210be45165652ee3a41
I20260808 07:07:18.678793 318 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dd8c626bbc4210be45165652ee3a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:07:18 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:41058
I20260808 07:07:18.678861 32455 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dd8c626bbc4210be45165652ee3a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:07:18 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:44794
I20260808 07:07:18.679068 449 tablet_replica.cc:330] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: stopping tablet replica
I20260808 07:07:18.679081 32622 tablet_service.cc:1514] Processing DeleteTablet for tablet 29dd8c626bbc4210be45165652ee3a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:07:18 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:50932
I20260808 07:07:18.679188 449 raft_consensus.cc:2229] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:07:18.679324 449 raft_consensus.cc:2258] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:07:18.679330 452 tablet_replica.cc:330] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: stopping tablet replica
I20260808 07:07:18.679473 452 raft_consensus.cc:2229] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:07:18.679631 452 raft_consensus.cc:2258] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:07:18.679754 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 32369
I20260808 07:07:18.679893 449 ts_tablet_manager.cc:1905] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:07:18.682449 449 ts_tablet_manager.cc:1918] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:07:18.682539 449 log.cc:1196] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/29dd8c626bbc4210be45165652ee3a41
I20260808 07:07:18.682644 452 ts_tablet_manager.cc:1905] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:07:18.682891 449 ts_tablet_manager.cc:1939] T 29dd8c626bbc4210be45165652ee3a41 P e7045dfbebb045f0b7aac6556e997306: Deleting consensus metadata
I20260808 07:07:18.683318 32270 catalog_manager.cc:4841] TS e7045dfbebb045f0b7aac6556e997306 (127.18.63.67:33701): tablet 29dd8c626bbc4210be45165652ee3a41 (table test-table [id=da927a092bc54dddaaf073f6ad3af889]) successfully deleted
I20260808 07:07:18.684306 452 ts_tablet_manager.cc:1918] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:07:18.684404 452 log.cc:1196] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/29dd8c626bbc4210be45165652ee3a41
I20260808 07:07:18.684723 452 ts_tablet_manager.cc:1939] T 29dd8c626bbc4210be45165652ee3a41 P 247907a7adb843e3b9a05befbbfd55ee: Deleting consensus metadata
I20260808 07:07:18.686625 32271 catalog_manager.cc:4841] TS 247907a7adb843e3b9a05befbbfd55ee (127.18.63.66:45711): tablet 29dd8c626bbc4210be45165652ee3a41 (table test-table [id=da927a092bc54dddaaf073f6ad3af889]) successfully deleted
W20260808 07:07:18.693951 32273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:41437 (error 108)
I20260808 07:07:18.694223 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 32544
I20260808 07:07:18.694990 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:07:18.694406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41437 (local address 127.0.0.1:44802)
0808 07:07:18.694875 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 07:07:18.694924 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442}
W20260808 07:07:18.695134 32273 catalog_manager.cc:4584] TS f84a23017e6548538d344c6eecc56502 (127.18.63.65:41437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29dd8c626bbc4210be45165652ee3a41: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41437: connect: Connection refused (error 111)
I20260808 07:07:18.703765 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 32708
I20260808 07:07:18.711688 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 32252
2026-08-08T07:07:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2154 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[457](info): set up 2 sockets
Aug 08 07:07:18 dist-test-slave-bw88 krb5kdc[457](info): commencing operation
krb5kdc: starting...
W20260808 07:07:19.910036 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:07:19 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:07:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangeradmin_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangeradmin_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangerlookup_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangerlookup_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
I20260808 07:07:20.086454 18685 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskERtyUc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/postgres -l logfile start
2026-08-08 07:07:22.605 UTC [491] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:07:22.605 UTC [491] LOG: listening on IPv4 address "127.18.63.84", port 56867
2026-08-08 07:07:22.608 UTC [491] LOG: listening on Unix socket "/tmp/.s.PGSQL.56867"
2026-08-08 07:07:22.644 UTC [495] LOG: database system was shut down at 2026-08-08 07:07:21 UTC
2026-08-08 07:07:22.648 UTC [491] LOG: database system is ready to accept connections
I20260808 07:07:23.762089 18685 mini_postgres.cc:96] Postgres bound to 56867
2026-08-08 07:07:23.770 UTC [505] FATAL: database "slave" does not exist
127.18.63.84:56867 - accepting connections
I20260808 07:07:23.770574 18685 mini_ranger.cc:161] Starting Ranger...
I20260808 07:07:23.786077 18685 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:07:24.235332 18685 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:07:24.235440 18685 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin
2026-08-08 07:07:24,518 [I] DB FLAVOR :POSTGRES
2026-08-08 07:07:24,519 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:07:24,519 [I] Checking connection..
2026-08-08 07:07:24,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:07:24,789 [I] Checking connection passed.
2026-08-08 07:07:24,789 [I] --------- Verifying version history table ---------
2026-08-08 07:07:24,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:07:25,034 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:07:25,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:07:25,299 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:07:25,299 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:07:25,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:07:25.563 UTC [582] WARNING: there is no transaction in progress
2026-08-08 07:07:25,572 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:07:25,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:07:25,840 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:07:25,840 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:07:25,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:07:26,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:07:26,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bw88.c.gcp-upstream.internal','N') ;"
2026-08-08 07:07:26,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:07:26,869 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:07:26,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:07:27,129 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:07:27,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:07:27,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:07:27,667 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:07:27,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:07:28.632 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.647 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.663 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.677 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.697 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.713 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.767 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.774 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.788 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.797 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.807 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.820 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.829 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.835 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.841 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:28.848 UTC [722] WARNING: there is no transaction in progress
2026-08-08 07:07:29,484 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:07:29,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:07:29,748 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:07:29,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:07:30,019 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:07:30,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:07:30,281 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:07:30,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:07:30,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:07:30,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bw88.c.gcp-upstream.internal';"
2026-08-08 07:07:31,061 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:07:31,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:07:31,310 [I] DB_PATCHES have already been applied
I20260808 07:07:31.316609 18685 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskERtyUc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:07:31.316677 18685 mini_ranger.cc:193] Using host: 127.18.63.84
I20260808 07:07:31.320470 18685 mini_ranger.cc:227] Ranger admin URL: http://127.18.63.84:37609
Aug 08, 2026 7:07:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:07:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:07:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:07:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:07:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:07:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:07:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:07:32.582933 18685 mini_ranger.cc:160] Time spent starting Ranger: real 8.812s user 0.000s sys 0.011s
Aug 08, 2026 7:07:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:07:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 7:07:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:07:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:07:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:07:55.246953 18685 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangerkms_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/rangerkms_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:07:55 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172875, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:07:55.332491 18685 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:07:55.342772 18685 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:07:56.206952 18685 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:07:56.207043 18685 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms
I20260808 07:07:56.207062 18685 mini_ranger_kms.cc:227] Using postgres at 127.18.63.84:56867
2026-08-08 07:07:56,813 [I] DB FLAVOR :POSTGRES
2026-08-08 07:07:56,813 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:07:56,813 [I] Checking connection
2026-08-08 07:07:56,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:07:57,104 [I] connection success
2026-08-08 07:07:57,105 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:07:57,105 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:07:57,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:07:57,373 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:07:57,374 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:07:57,374 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:07:57,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:56867/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:07:57,689 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:07:58.192368 18685 mini_ranger_kms.cc:310] Created kms service
I20260808 07:07:58.408552 18685 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:07:58.454885 18685 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:07:58 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172875, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:07:58.885732 18685 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:07:58.885866 18685 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms:/tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:07:58.885891 18685 mini_ranger_kms.cc:240] Using host: 127.18.63.84
I20260808 07:07:58.891124 18685 mini_ranger_kms.cc:276] Ranger KMS PID: 962
I20260808 07:07:58.891254 18685 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.63.84:36537
Aug 08, 2026 7:08:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:08:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:08:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:08:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:08:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:08:01.614430 18685 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.282s user 0.000s sys 0.579s
I20260808 07:08:01.614553 18685 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:08:01.614647 18685 mini_ranger_kms.cc:401] 127.18.63.84:36537/kms/v1/keys
Aug 08 07:08:01 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172881, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:08:02 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172881, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
Aug 08 07:08:02 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172881, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
Aug 08 07:08:05 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:08:05.444087 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44119
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:44119
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:36537/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:05.552809 1020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:05.553089 1020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:05.553133 1020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:05.554394 1020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:08:05.554454 1020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:05.554477 1020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:05.554498 1020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:08:05.554517 1020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:08:05.556344 1020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.63.84:36537/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:44119
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:05.556640 1020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:05.556903 1020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:05.562439 1029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:05.562485 1031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:05.562604 1027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:05.563130 1020 server_base.cc:1034] running on GCE node
I20260808 07:08:05.563436 1020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:05.563787 1020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:05.565011 1020 hybrid_clock.cc:648] HybridClock initialized: now 1786172885564981 us; error 47 us; skew 500 ppm
Aug 08 07:08:05 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:05.568240 1020 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:08:05.568846 1020 webserver.cc:460] Webserver started at http://127.18.63.126:46451/ using document root <none> and password file <none>
I20260808 07:08:05.569123 1020 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:05.569170 1020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:05.569319 1020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:05 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:05.622531 1020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "8e2cb7b7439d415e859f5fc15b620de7"
format_stamp: "Formatted at 2026-08-08 07:08:05 on dist-test-slave-bw88"
server_key: "1c6f8ee8089baad8bce8c67e6c0e2ae3"
server_key_iv: "713022f946bc09c3bfe3763c571d12b4"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:05.623035 1020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "8e2cb7b7439d415e859f5fc15b620de7"
format_stamp: "Formatted at 2026-08-08 07:08:05 on dist-test-slave-bw88"
server_key: "1c6f8ee8089baad8bce8c67e6c0e2ae3"
server_key_iv: "713022f946bc09c3bfe3763c571d12b4"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:05.624924 1020 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 07:08:05.638180 1044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:05.638429 1020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:08:05.638545 1020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8e2cb7b7439d415e859f5fc15b620de7"
format_stamp: "Formatted at 2026-08-08 07:08:05 on dist-test-slave-bw88"
server_key: "1c6f8ee8089baad8bce8c67e6c0e2ae3"
server_key_iv: "713022f946bc09c3bfe3763c571d12b4"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:05.638679 1020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:05.654465 1020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:05.656804 1020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:05.657006 1020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:05.663347 1020 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:44119
I20260808 07:08:05.663383 1116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:44119 every 8 connection(s)
I20260808 07:08:05.663753 1020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:08:05.664494 1117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:05.667117 1117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: Bootstrap starting.
I20260808 07:08:05.667732 1117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:05.668043 1117 log.cc:824] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:05.669039 1117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: No bootstrap required, opened a new log
I20260808 07:08:05.671211 1117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } }
I20260808 07:08:05.671367 1117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:05.671389 1117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2cb7b7439d415e859f5fc15b620de7, State: Initialized, Role: FOLLOWER
I20260808 07:08:05.671561 1117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } }
I20260808 07:08:05.671645 1117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:08:05.671676 1117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:08:05.671723 1117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:05.671967 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 1020
I20260808 07:08:05.672088 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:08:05.672298 18685 external_mini_cluster.cc:1428] Setting key 3645a4c222b180f296c2ec54462400c9
I20260808 07:08:05.672555 1117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } }
I20260808 07:08:05.672757 1117 leader_election.cc:302] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e2cb7b7439d415e859f5fc15b620de7; no voters:
I20260808 07:08:05.672986 1117 leader_election.cc:288] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:08:05.673135 1124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:08:05.673375 1117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:08:05.673403 1124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [term 1 LEADER]: Becoming Leader. State: Replica: 8e2cb7b7439d415e859f5fc15b620de7, State: Running, Role: LEADER
I20260808 07:08:05.673615 1124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } }
I20260808 07:08:05.674049 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e2cb7b7439d415e859f5fc15b620de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } } }
I20260808 07:08:05.674121 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:05.674356 1126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e2cb7b7439d415e859f5fc15b620de7. Latest consensus state: current_term: 1 leader_uuid: "8e2cb7b7439d415e859f5fc15b620de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2cb7b7439d415e859f5fc15b620de7" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44119 } } }
I20260808 07:08:05.674454 1126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7 [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:05.674970 1134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:08:05.675330 1117 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:08:05.675521 1134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:08:05.682392 1134 catalog_manager.cc:1347] Generated new cluster ID: ad33bce8b13a4d439b7430e5c361f68c
I20260808 07:08:05.682685 1134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:08:05 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:05.696942 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:05.672806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.18.63.126:44119)
0808 07:08:05.673043 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:05.673054 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:05.678179 (+ 5125us) server_negotiation.cc:407] Connection header received
0808 07:08:05.678805 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:05.678814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:05.678946 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:05.679097 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:05.680393 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:05.683112 (+ 2719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:05.687682 (+ 4570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:05.689248 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:05.689378 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:05.692367 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:05.692372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:05.692372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:05.692385 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:05.694642 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:05.695016 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:05.695020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:05.695020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:05.695126 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:05.695331 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:05.695332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:05.695332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:05.695730 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:05.695844 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:05.696216 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 07:08:05.696519 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":178,"thread_start_us":113,"threads_started":1}
I20260808 07:08:05.735514 1134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:08:05.736552 1134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:08:05.743754 1134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: Generated new TSK 0
I20260808 07:08:05.744122 1134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:08:06.810551 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:36537/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:06.940708 1176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:06.940953 1176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:06.940997 1176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:06.942441 1176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:06.942507 1176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:06.942553 1176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:08:06.944304 1176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.63.84:36537/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:06.944571 1176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:06.944854 1176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:06.950827 1182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:06.950865 1183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:06.951215 1176 server_base.cc:1034] running on GCE node
W20260808 07:08:06.951454 1185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:06.951709 1176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:06.952005 1176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:06.953162 1176 hybrid_clock.cc:648] HybridClock initialized: now 1786172886953152 us; error 28 us; skew 500 ppm
Aug 08 07:08:06 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:06.957051 1176 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:08:06.958048 1176 webserver.cc:460] Webserver started at http://127.18.63.65:36109/ using document root <none> and password file <none>
I20260808 07:08:06.958407 1176 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:06.958515 1176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:06.958669 1176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:06 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:06.974808 1176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "44751026e507439d86c3f3a4cfb005be"
format_stamp: "Formatted at 2026-08-08 07:08:06 on dist-test-slave-bw88"
server_key: "13d2ebcd895de5cf41802229ea2cc349"
server_key_iv: "40fbdfb0a0aa6b15d7351935a78fe65b"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:06.975346 1176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "44751026e507439d86c3f3a4cfb005be"
format_stamp: "Formatted at 2026-08-08 07:08:06 on dist-test-slave-bw88"
server_key: "13d2ebcd895de5cf41802229ea2cc349"
server_key_iv: "40fbdfb0a0aa6b15d7351935a78fe65b"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:06.977514 1176 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:08:06.990758 1193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:06.991021 1176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:08:06.991142 1176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "44751026e507439d86c3f3a4cfb005be"
format_stamp: "Formatted at 2026-08-08 07:08:06 on dist-test-slave-bw88"
server_key: "13d2ebcd895de5cf41802229ea2cc349"
server_key_iv: "40fbdfb0a0aa6b15d7351935a78fe65b"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:06.991258 1176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:07.028276 1176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:07.030866 1176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:07.031122 1176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:07.031447 1176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:07.031905 1176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:07.031998 1176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.032102 1176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:07.032176 1176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.051702 1176 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:35373
I20260808 07:08:07.052321 1176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:08:07.058423 1342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:35373 every 8 connection(s)
I20260808 07:08:07.060091 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 1176
I20260808 07:08:07.060213 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:08:07.060451 18685 external_mini_cluster.cc:1428] Setting key 39f8c1e7a377cfe56baa0803c006e963
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172886, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:07.074926 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.059881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:54529 (local address 127.18.63.126:44119)
0808 07:08:07.061571 (+ 1690us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.061574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.061582 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:08:07.061607 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.061608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.061621 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.063100 (+ 1479us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:07.063168 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.064092 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.066724 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.066917 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.067014 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.070692 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:07.070693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:07.070694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:07.070702 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:07.073033 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.073649 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.073650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.073651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.073718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.074027 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.074028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.074028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.074193 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:07.074255 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.074693 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.074791 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1666,"thread_start_us":149,"threads_started":1}
I20260808 07:08:07.075814 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.059325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44119 (local address 127.18.63.65:54529)
0808 07:08:07.059728 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.059776 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.061373 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.061868 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.061873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.062437 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:07.062988 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.062996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.064214 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.064215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.066629 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.066631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.067019 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.067020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.067320 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.068131 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:07.068142 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:07.070574 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:07.073154 (+ 2580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.073162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.073165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.073548 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.073795 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.073796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.073796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.073941 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.074294 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:07.074300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:07.074483 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.075292 (+ 809us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.075486 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":262,"threads_started":1}
I20260808 07:08:07.076647 1343 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44119
I20260808 07:08:07.076773 1343 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:07.076972 1343 heartbeater.cc:511] Master 127.18.63.126:44119 requested a full tablet report, sending...
I20260808 07:08:07.077560 1065 ts_manager.cc:194] Registered new tserver with Master: 44751026e507439d86c3f3a4cfb005be (127.18.63.65:35373)
I20260808 07:08:07.078171 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:54529
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:08:07.117275 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:36537/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:07.294027 1350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:07.294417 1350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:07.294540 1350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:07.296550 1350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:07.296695 1350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:07.296799 1350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:08:07.299540 1350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.63.84:36537/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:07.300050 1350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:07.300401 1350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:07.315400 1358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:07.315867 1357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:07.316975 1360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:07.318161 1350 server_base.cc:1034] running on GCE node
I20260808 07:08:07.318427 1350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:07.318735 1350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:07.326073 1350 hybrid_clock.cc:648] HybridClock initialized: now 1786172887326040 us; error 52 us; skew 500 ppm
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:07.330513 1350 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:08:07.331117 1350 webserver.cc:460] Webserver started at http://127.18.63.66:43363/ using document root <none> and password file <none>
I20260808 07:08:07.331430 1350 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:07.331497 1350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:07.331650 1350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:07.349817 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "3f8ea2e03505431ea1bcaad062e39a7c"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "ffb1c910a5c75ccfe462f0d50c014495"
server_key_iv: "0213d8bca5a96c4a354542ac47f3277e"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.350382 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "3f8ea2e03505431ea1bcaad062e39a7c"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "ffb1c910a5c75ccfe462f0d50c014495"
server_key_iv: "0213d8bca5a96c4a354542ac47f3277e"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.353591 1350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 07:08:07.371747 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.371970 1350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:08:07.372089 1350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "3f8ea2e03505431ea1bcaad062e39a7c"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "ffb1c910a5c75ccfe462f0d50c014495"
server_key_iv: "0213d8bca5a96c4a354542ac47f3277e"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.372202 1350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:07.393385 1350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:07.395851 1350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:07.396030 1350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:07.396409 1350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:07.396864 1350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:07.396914 1350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.396942 1350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:07.396965 1350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.417261 1350 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:39441
I20260808 07:08:07.417840 1350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:08:07.420266 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 1350
I20260808 07:08:07.420392 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:08:07.420622 18685 external_mini_cluster.cc:1428] Setting key d59be33a8fed76e5ce48daff262b6ebf
I20260808 07:08:07.427596 1520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:39441 every 8 connection(s)
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:07.452208 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.429087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43167 (local address 127.18.63.126:44119)
0808 07:08:07.429159 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.429161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.429385 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:08:07.430342 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.430344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.430361 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.430421 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:07.434558 (+ 4137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.435415 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.437735 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.438139 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.438222 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.447535 (+ 9313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:07.447537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:07.447538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:07.447547 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:07.449316 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.450094 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.450096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.450096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.450170 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.451326 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.451328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.451328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.451490 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:07.451564 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.451978 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.452097 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:08:07.452450 1526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.428669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44119 (local address 127.18.63.66:43167)
0808 07:08:07.429204 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.429253 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.430226 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.433496 (+ 3270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.433501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.433961 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:07.434396 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.434404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.435594 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.435596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.437626 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.437630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.438249 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.438250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.438484 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.439356 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:07.439367 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:07.447377 (+ 8010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:07.449479 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.449490 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.449493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.449902 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.451049 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.451051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.451051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.451238 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.451572 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:07.451581 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:07.451814 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.451941 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.452143 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":204,"threads_started":1}
I20260808 07:08:07.453038 1521 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44119
I20260808 07:08:07.453155 1521 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:07.453418 1521 heartbeater.cc:511] Master 127.18.63.126:44119 requested a full tablet report, sending...
I20260808 07:08:07.453858 1065 ts_manager.cc:194] Registered new tserver with Master: 3f8ea2e03505431ea1bcaad062e39a7c (127.18.63.66:39441)
I20260808 07:08:07.454409 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:43167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:08:07.464672 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:36537/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:44873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:07.634088 1528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:07.634482 1528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:07.634582 1528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:07.636567 1528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:07.636693 1528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:07.636783 1528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:08:07.639427 1528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:44873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.63.84:36537/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:44119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:07.639950 1528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:07.640309 1528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:07.654599 1540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:07.654635 1536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:07.655148 1542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:07.655682 1528 server_base.cc:1034] running on GCE node
I20260808 07:08:07.656091 1528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:07.656397 1528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:07.659684 1528 hybrid_clock.cc:648] HybridClock initialized: now 1786172887659655 us; error 54 us; skew 500 ppm
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:07.664053 1528 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:08:07.664790 1528 webserver.cc:460] Webserver started at http://127.18.63.67:37537/ using document root <none> and password file <none>
I20260808 07:08:07.665130 1528 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:07.665262 1528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:07.665503 1528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:07.693722 1528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "35b6494d8eda4740a114d36de59980a6"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "cc5fef8fbfe8e6ab73f13b3396b29580"
server_key_iv: "fa1684f0c30b6edc62fd839343e8ea9d"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.694406 1528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "35b6494d8eda4740a114d36de59980a6"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "cc5fef8fbfe8e6ab73f13b3396b29580"
server_key_iv: "fa1684f0c30b6edc62fd839343e8ea9d"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.706305 1528 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.001s sys 0.001s
I20260808 07:08:07.724305 1549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.724625 1528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:08:07.724843 1528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "35b6494d8eda4740a114d36de59980a6"
format_stamp: "Formatted at 2026-08-08 07:08:07 on dist-test-slave-bw88"
server_key: "cc5fef8fbfe8e6ab73f13b3396b29580"
server_key_iv: "fa1684f0c30b6edc62fd839343e8ea9d"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:07.725015 1528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:07.824545 1528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:07.828958 1528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:07.829375 1528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:07.829928 1528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:07.830935 1528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:07.831048 1528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.831133 1528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:07.831168 1528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:07.878091 1528 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:45523
I20260808 07:08:07.878717 1528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:08:07.881374 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 1528
I20260808 07:08:07.881497 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:08:07.881700 18685 external_mini_cluster.cc:1428] Setting key e675c5a595c2cc8159db1119bc98bfaa
I20260808 07:08:07.885936 1697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:45523 every 8 connection(s)
Aug 08 07:08:07 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:07.904660 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.879877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:56541 (local address 127.18.63.126:44119)
0808 07:08:07.879938 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.879940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.889886 (+ 9946us) server_negotiation.cc:407] Connection header received
0808 07:08:07.890678 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.890680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.890693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.890767 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:07.892057 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.892863 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.895701 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.896002 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.896052 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.899677 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:07.899679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:07.899680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:07.899688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:07.901486 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.902398 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.902400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.902400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.902464 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.903285 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.903286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.903287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.903432 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:07.903488 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.904465 (+ 977us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.904554 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:08:07.905697 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.879924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44119 (local address 127.18.63.67:56541)
0808 07:08:07.889441 (+ 9517us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.889580 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.890551 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.890918 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.890923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.891379 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:07.891747 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.891754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.894227 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.894228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.895569 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.895572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.896152 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.896152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.896357 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.897154 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:07.897165 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:07.899554 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:07.901979 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.901986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.901988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.902300 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.902541 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:07.902542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:07.902542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:07.902673 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:07.903525 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:07.903532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:07.904286 (+ 754us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.905251 (+ 965us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.905446 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9457,"thread_start_us":139,"threads_started":1}
I20260808 07:08:07.906641 1700 heartbeater.cc:347] Connected to a master server at 127.18.63.126:44119
I20260808 07:08:07.906756 1700 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:07.906960 1700 heartbeater.cc:511] Master 127.18.63.126:44119 requested a full tablet report, sending...
I20260808 07:08:07.907337 1065 ts_manager.cc:194] Registered new tserver with Master: 35b6494d8eda4740a114d36de59980a6 (127.18.63.67:45523)
I20260808 07:08:07.907819 1065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:56541
I20260808 07:08:07.914871 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:08:07.935281 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.923828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.18.63.126:44119)
0808 07:08:07.923875 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.923877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.924813 (+ 936us) server_negotiation.cc:407] Connection header received
0808 07:08:07.924924 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.924925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.924937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.924968 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:07.926101 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.926911 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.929068 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.929380 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.929431 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.931885 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:07.931887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:07.931887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:07.931896 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:07.933748 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.934178 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.934179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.934179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.934243 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:07.934654 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:07.934655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:07.934655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:07.934807 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:07.934879 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.935098 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.935182 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:08:07.937770 1065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:08:07.940268 1065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:08:07.959594 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.945902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:45523 (local address 127.0.0.1:35150)
0808 07:08:07.951571 (+ 5669us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.951580 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.951626 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.952279 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.952283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.952286 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:07.952403 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.952405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.955234 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.955235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.957871 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.957874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.959156 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.959157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.959243 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.959258 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.959329 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.959387 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5647,"mutex_wait_us":1416,"thread_start_us":383,"threads_started":1}
I20260808 07:08:07.960800 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.947043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:39441 (local address 127.0.0.1:50888)
0808 07:08:07.955516 (+ 8473us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.955528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.955726 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.956108 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.956109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.956111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:07.956145 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.956146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.957821 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.957821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.958953 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.958955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.960567 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.960568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.960632 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.960638 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.960685 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.960726 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8428,"thread_start_us":241,"threads_started":1}
I20260808 07:08:07.960983 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.945567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:35373 (local address 127.0.0.1:46356)
0808 07:08:07.951172 (+ 5605us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:07.951218 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:07.951396 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:07.953804 (+ 2408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:07.953805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:07.953807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:07.953850 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.953852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.955234 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.955235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.956732 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:07.956735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.960756 (+ 4021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.960756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:07.960811 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.960815 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:07.960865 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:08:07.960907 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5553,"mutex_wait_us":3,"thread_start_us":185,"threads_started":1}
I20260808 07:08:07.961215 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.947471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.18.63.66:39441)
0808 07:08:07.951096 (+ 3625us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.951102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.955794 (+ 4692us) server_negotiation.cc:407] Connection header received
0808 07:08:07.955862 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.955864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.955881 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.956033 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:07.956214 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.957730 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.959062 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.960409 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.960476 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.960531 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.960982 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.961109 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3585,"thread_start_us":153,"threads_started":1}
I20260808 07:08:07.961589 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.945700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.18.63.65:35373)
0808 07:08:07.953537 (+ 7837us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.953543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.953555 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:08:07.953596 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.953598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.953616 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.953715 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:07.953951 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.955135 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.959809 (+ 4674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.960605 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.960671 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.960724 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.961399 (+ 675us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.961460 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7794,"thread_start_us":62,"threads_started":1}
I20260808 07:08:07.962073 1439 tablet_service.cc:1467] Processing CreateTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 (DEFAULT_TABLE table=test-table [id=8354d787bfb946f38d5807e78891db12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:07.962242 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 (DEFAULT_TABLE table=test-table [id=8354d787bfb946f38d5807e78891db12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:07.962519 1439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d245e8243a4fb1ad6b5fe187581653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:07.962555 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d245e8243a4fb1ad6b5fe187581653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:07.961313 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:07.946528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.18.63.67:45523)
0808 07:08:07.951792 (+ 5264us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:07.951797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:07.951808 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:08:07.951854 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:07.951856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:07.951873 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:07.952134 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:07.952504 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.955127 (+ 2623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.957989 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:07.958859 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:07.958935 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:07.959121 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:07.961099 (+ 1978us) server_negotiation.cc:299] Negotiation successful
0808 07:08:07.961146 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5218,"thread_start_us":224,"threads_started":1}
I20260808 07:08:07.968108 1602 tablet_service.cc:1467] Processing CreateTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 (DEFAULT_TABLE table=test-table [id=8354d787bfb946f38d5807e78891db12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:07.968546 1602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d245e8243a4fb1ad6b5fe187581653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:07.975116 1727 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Bootstrap starting.
I20260808 07:08:07.976977 1726 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Bootstrap starting.
I20260808 07:08:07.979429 1726 tablet_bootstrap.cc:654] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:07.980352 1726 log.cc:824] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:07.980846 1725 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Bootstrap starting.
I20260808 07:08:07.982414 1725 tablet_bootstrap.cc:654] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:07.983225 1725 log.cc:824] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:07.983639 1727 tablet_bootstrap.cc:654] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:07.984473 1727 log.cc:824] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:07.986127 1726 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: No bootstrap required, opened a new log
I20260808 07:08:07.986203 1726 ts_tablet_manager.cc:1397] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260808 07:08:07.990360 1725 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: No bootstrap required, opened a new log
I20260808 07:08:07.990430 1725 ts_tablet_manager.cc:1397] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260808 07:08:07.994717 1725 raft_consensus.cc:348] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:07.994922 1725 raft_consensus.cc:374] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:07.995770 1725 raft_consensus.cc:729] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44751026e507439d86c3f3a4cfb005be, State: Initialized, Role: FOLLOWER
I20260808 07:08:07.995796 1726 raft_consensus.cc:348] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:07.996030 1725 consensus_queue.cc:260] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:07.996105 1726 raft_consensus.cc:374] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:07.996148 1726 raft_consensus.cc:729] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b6494d8eda4740a114d36de59980a6, State: Initialized, Role: FOLLOWER
I20260808 07:08:07.996296 1726 consensus_queue.cc:260] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:07.996512 1725 ts_tablet_manager.cc:1428] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260808 07:08:07.996654 1343 heartbeater.cc:503] Master 127.18.63.126:44119 was elected leader, sending a full tablet report...
I20260808 07:08:07.997370 1726 ts_tablet_manager.cc:1428] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Time spent starting tablet: real 0.011s user 0.003s sys 0.002s
I20260808 07:08:07.997541 1700 heartbeater.cc:503] Master 127.18.63.126:44119 was elected leader, sending a full tablet report...
I20260808 07:08:07.998329 1727 tablet_bootstrap.cc:492] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: No bootstrap required, opened a new log
I20260808 07:08:07.998425 1727 ts_tablet_manager.cc:1397] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260808 07:08:08.003037 1727 raft_consensus.cc:348] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:08.003221 1727 raft_consensus.cc:374] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:08.003252 1727 raft_consensus.cc:729] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f8ea2e03505431ea1bcaad062e39a7c, State: Initialized, Role: FOLLOWER
I20260808 07:08:08.003376 1727 consensus_queue.cc:260] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:08.003726 1727 ts_tablet_manager.cc:1428] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 07:08:08.003867 1521 heartbeater.cc:503] Master 127.18.63.126:44119 was elected leader, sending a full tablet report...
I20260808 07:08:08.038686 1734 raft_consensus.cc:482] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:08:08.038913 1734 raft_consensus.cc:504] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:08.039521 1734 leader_election.cc:288] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b6494d8eda4740a114d36de59980a6 (127.18.63.67:45523), 3f8ea2e03505431ea1bcaad062e39a7c (127.18.63.66:39441)
I20260808 07:08:08.046525 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.041685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:48037 (local address 127.18.63.67:45523)
0808 07:08:08.041782 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:08.041785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:08.041795 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:08:08.041830 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:08.041831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:08.041840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:08.041939 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:08.042538 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.043633 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.045471 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.046064 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.046113 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:08.046123 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:08.046407 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 07:08:08.046454 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:08:08.049899 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.039657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:45523 (local address 127.18.63.65:48037)
0808 07:08:08.040899 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:08.040921 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:08.041278 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:08.042053 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:08.042064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:08.042069 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:08.042428 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:08.042432 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.043923 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.043924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:08.045366 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:08.045370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.046184 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.046184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:08.046320 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:08.046324 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:08.046726 (+ 402us) client_negotiation.cc:240] Negotiation successful
0808 07:08:08.049765 (+ 3039us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"thread_start_us":300,"threads_started":1}
I20260808 07:08:08.050455 1640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d245e8243a4fb1ad6b5fe187581653" candidate_uuid: "44751026e507439d86c3f3a4cfb005be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b6494d8eda4740a114d36de59980a6" is_pre_election: true
I20260808 07:08:08.050655 1640 raft_consensus.cc:2454] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44751026e507439d86c3f3a4cfb005be in term 0.
I20260808 07:08:08.051220 1195 leader_election.cc:302] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b6494d8eda4740a114d36de59980a6, 44751026e507439d86c3f3a4cfb005be; no voters:
I20260808 07:08:08.051407 1734 raft_consensus.cc:2781] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:08:08.051499 1734 raft_consensus.cc:482] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:08:08.051527 1734 raft_consensus.cc:3037] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:08.053181 1734 raft_consensus.cc:504] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:08.053550 1640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d245e8243a4fb1ad6b5fe187581653" candidate_uuid: "44751026e507439d86c3f3a4cfb005be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b6494d8eda4740a114d36de59980a6"
I20260808 07:08:08.053650 1640 raft_consensus.cc:3037] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:08.054261 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.040448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:55175 (local address 127.18.63.66:39441)
0808 07:08:08.040505 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:08.040508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:08.049690 (+ 9182us) server_negotiation.cc:407] Connection header received
0808 07:08:08.050181 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:08.050183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:08.050193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:08.050221 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:08.050532 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.051219 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.053032 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.053717 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.053765 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:08.053774 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:08.054140 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 07:08:08.054190 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:08:08.054807 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.039834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:39441 (local address 127.18.63.65:55175)
0808 07:08:08.049475 (+ 9641us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:08.049487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:08.050085 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:08.050281 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:08.050282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:08.050283 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:08.050450 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:08.050452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.051454 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.051455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:08.052915 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:08.052918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.053916 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.053917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:08.054043 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:08.054047 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:08.054500 (+ 453us) client_negotiation.cc:240] Negotiation successful
0808 07:08:08.054543 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9578,"thread_start_us":257,"threads_started":1}
I20260808 07:08:08.055429 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d245e8243a4fb1ad6b5fe187581653" candidate_uuid: "44751026e507439d86c3f3a4cfb005be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" is_pre_election: true
I20260808 07:08:08.055516 1640 raft_consensus.cc:2454] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44751026e507439d86c3f3a4cfb005be in term 1.
I20260808 07:08:08.055574 1466 raft_consensus.cc:2454] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44751026e507439d86c3f3a4cfb005be in term 0.
I20260808 07:08:08.055711 1195 leader_election.cc:302] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b6494d8eda4740a114d36de59980a6, 44751026e507439d86c3f3a4cfb005be; no voters:
I20260808 07:08:08.056077 1734 leader_election.cc:288] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [CANDIDATE]: Term 1 election: Requested vote from peers 35b6494d8eda4740a114d36de59980a6 (127.18.63.67:45523), 3f8ea2e03505431ea1bcaad062e39a7c (127.18.63.66:39441)
I20260808 07:08:08.056128 1734 raft_consensus.cc:2781] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:08:08.056167 1734 raft_consensus.cc:686] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 1 LEADER]: Becoming Leader. State: Replica: 44751026e507439d86c3f3a4cfb005be, State: Running, Role: LEADER
I20260808 07:08:08.056245 1734 consensus_queue.cc:237] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } }
I20260808 07:08:08.057154 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d245e8243a4fb1ad6b5fe187581653" candidate_uuid: "44751026e507439d86c3f3a4cfb005be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f8ea2e03505431ea1bcaad062e39a7c"
I20260808 07:08:08.057235 1466 raft_consensus.cc:3037] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:08.057423 1065 catalog_manager.cc:5495] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be reported cstate change: term changed from 0 to 1, leader changed from <none> to 44751026e507439d86c3f3a4cfb005be (127.18.63.65). New cstate: current_term: 1 leader_uuid: "44751026e507439d86c3f3a4cfb005be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44751026e507439d86c3f3a4cfb005be" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 35373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20260808 07:08:08.058847 1466 raft_consensus.cc:2454] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44751026e507439d86c3f3a4cfb005be in term 1.
W20260808 07:08:08.060316 1344 tablet.cc:2369] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 07:08:08 dist-test-slave-bw88 krb5kdc[457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172887, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:08.071684 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.067957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.18.63.65:35373)
0808 07:08:08.068000 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:08.068002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:08.068009 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:08:08.068041 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:08.068042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:08.068062 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:08.068145 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:08:08.068357 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.069290 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.070508 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:08.070801 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:08.070872 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:08.071184 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:08:08.071320 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:08:08.071401 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:08.071584 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:08:08.071604 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:08:08.074883 1466 raft_consensus.cc:1261] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 1 FOLLOWER]: Refusing update from remote peer 44751026e507439d86c3f3a4cfb005be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:08:08.074919 1640 raft_consensus.cc:1261] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 1 FOLLOWER]: Refusing update from remote peer 44751026e507439d86c3f3a4cfb005be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:08:08.075371 1742 consensus_queue.cc:1035] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b6494d8eda4740a114d36de59980a6" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:08:08.075490 1734 consensus_queue.cc:1035] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f8ea2e03505431ea1bcaad062e39a7c" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:08:08.080785 1745 mvcc.cc:205] Tried to move back new op lower bound from 7316164149549764608 to 7316164149478350848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164149549764608}]
I20260808 07:08:08.081188 1746 mvcc.cc:205] Tried to move back new op lower bound from 7316164149549764608 to 7316164149478350848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164149549764608}]
I20260808 07:08:08.081727 1750 mvcc.cc:205] Tried to move back new op lower bound from 7316164149549764608 to 7316164149478350848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164149549764608}]
I20260808 07:08:08.085126 1065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:08:08.085224 1065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:08:08.086158 1065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e2cb7b7439d415e859f5fc15b620de7: Sending DeleteTablet for 3 replicas of tablet 83d245e8243a4fb1ad6b5fe187581653
I20260808 07:08:08.086796 1439 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:08 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:50888
I20260808 07:08:08.087236 1759 tablet_replica.cc:330] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: stopping tablet replica
I20260808 07:08:08.087353 1759 raft_consensus.cc:2229] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:08:08.087438 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:08 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:46356
I20260808 07:08:08.087488 1759 raft_consensus.cc:2258] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:08:08.087911 1761 tablet_replica.cc:330] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: stopping tablet replica
I20260808 07:08:08.088007 1761 raft_consensus.cc:2229] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:08:08.088148 1761 raft_consensus.cc:2258] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:08:08.089902 1602 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d245e8243a4fb1ad6b5fe187581653 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:08 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:35150
I20260808 07:08:08.090442 1761 ts_tablet_manager.cc:1905] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:08:08.090829 1759 ts_tablet_manager.cc:1905] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:08:08.092887 1761 ts_tablet_manager.cc:1918] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:08:08.093032 1761 log.cc:1196] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/83d245e8243a4fb1ad6b5fe187581653
I20260808 07:08:08.093403 1761 ts_tablet_manager.cc:1939] T 83d245e8243a4fb1ad6b5fe187581653 P 44751026e507439d86c3f3a4cfb005be: Deleting consensus metadata
I20260808 07:08:08.093479 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 1176
I20260808 07:08:08.095300 1759 ts_tablet_manager.cc:1918] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:08:08.095363 1759 log.cc:1196] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/83d245e8243a4fb1ad6b5fe187581653
I20260808 07:08:08.095600 1759 ts_tablet_manager.cc:1939] T 83d245e8243a4fb1ad6b5fe187581653 P 3f8ea2e03505431ea1bcaad062e39a7c: Deleting consensus metadata
I20260808 07:08:08.096189 1051 catalog_manager.cc:4841] TS 3f8ea2e03505431ea1bcaad062e39a7c (127.18.63.66:39441): tablet 83d245e8243a4fb1ad6b5fe187581653 (table test-table [id=8354d787bfb946f38d5807e78891db12]) successfully deleted
W20260808 07:08:08.108908 1047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:35373 (error 108)
I20260808 07:08:08.109660 1762 tablet_replica.cc:330] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: stopping tablet replica
I20260808 07:08:08.109822 1762 raft_consensus.cc:2229] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:08:08.109889 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:08.109757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:35373 (local address 127.0.0.1:46366)
0808 07:08:08.109811 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:08.109845 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:08:08.109970 1762 raft_consensus.cc:2258] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 07:08:08.110142 1047 catalog_manager.cc:4584] TS 44751026e507439d86c3f3a4cfb005be (127.18.63.65:35373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83d245e8243a4fb1ad6b5fe187581653: Network error: Client connection negotiation failed: client connection to 127.18.63.65:35373: connect: Connection refused (error 111)
I20260808 07:08:08.110558 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 1350
I20260808 07:08:08.119261 1762 ts_tablet_manager.cc:1905] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:08:08.123991 1762 ts_tablet_manager.cc:1918] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:08:08.124145 1762 log.cc:1196] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/83d245e8243a4fb1ad6b5fe187581653
I20260808 07:08:08.124851 1762 ts_tablet_manager.cc:1939] T 83d245e8243a4fb1ad6b5fe187581653 P 35b6494d8eda4740a114d36de59980a6: Deleting consensus metadata
I20260808 07:08:08.125460 1047 catalog_manager.cc:4841] TS 35b6494d8eda4740a114d36de59980a6 (127.18.63.67:45523): tablet 83d245e8243a4fb1ad6b5fe187581653 (table test-table [id=8354d787bfb946f38d5807e78891db12]) successfully deleted
I20260808 07:08:08.130963 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 1528
I20260808 07:08:08.153309 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 1020
I20260808 07:08:08.169298 18685 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:08:08.514041 18685 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:08:08.514153 18685 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:08:08.867777 18685 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:08:08.868 UTC [491] LOG: received smart shutdown request
2026-08-08 07:08:08.871 UTC [491] LOG: background worker "logical replication launcher" (PID 503) exited with exit code 1
2026-08-08 07:08:08.872 UTC [496] LOG: shutting down
2026-08-08 07:08:09.036 UTC [491] LOG: database system is shut down
2026-08-08T07:08:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50437 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:08:09 dist-test-slave-bw88 krb5kdc[1777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:08:09 dist-test-slave-bw88 krb5kdc[1777](info): set up 2 sockets
Aug 08 07:08:09 dist-test-slave-bw88 krb5kdc[1777](info): commencing operation
krb5kdc: starting...
W20260808 07:08:10.372596 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:08:10 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:08:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangeradmin_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangeradmin_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangerlookup_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangerlookup_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
I20260808 07:08:10.544090 18685 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskERtyUc/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/postgres -l logfile start
2026-08-08 07:08:13.016 UTC [1818] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 07:08:13.016 UTC [1818] LOG: listening on IPv4 address "127.18.63.84", port 59415
2026-08-08 07:08:13.020 UTC [1818] LOG: listening on Unix socket "/tmp/.s.PGSQL.59415"
2026-08-08 07:08:13.042 UTC [1821] LOG: database system was shut down at 2026-08-08 07:08:11 UTC
2026-08-08 07:08:13.046 UTC [1818] LOG: database system is ready to accept connections
I20260808 07:08:14.196780 18685 mini_postgres.cc:96] Postgres bound to 59415
2026-08-08 07:08:14.203 UTC [1831] FATAL: database "slave" does not exist
127.18.63.84:59415 - accepting connections
I20260808 07:08:14.204433 18685 mini_ranger.cc:161] Starting Ranger...
I20260808 07:08:14.219921 18685 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 07:08:14.680678 18685 mini_ranger.cc:87] Created ranger Postgres database
I20260808 07:08:14.680747 18685 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin
2026-08-08 07:08:14,971 [I] DB FLAVOR :POSTGRES
2026-08-08 07:08:14,972 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:08:14,972 [I] Checking connection..
2026-08-08 07:08:14,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 07:08:15,244 [I] Checking connection passed.
2026-08-08 07:08:15,244 [I] --------- Verifying version history table ---------
2026-08-08 07:08:15,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:08:15,511 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:08:15,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:08:15,778 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 07:08:15,778 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 07:08:15,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 07:08:16.015 UTC [1908] WARNING: there is no transaction in progress
2026-08-08 07:08:16,024 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 07:08:16,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 07:08:16,319 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 07:08:16,319 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:08:16,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 07:08:16,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 07:08:16,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bw88.c.gcp-upstream.internal','N') ;"
2026-08-08 07:08:17,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:08:17,380 [I] Table x_portal_user does not exist in database ranger
2026-08-08 07:08:17,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:08:17,646 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 07:08:17,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:08:17,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:08:18,167 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 07:08:18,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 07:08:19.157 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.176 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.193 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.212 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.231 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.252 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.317 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.324 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.337 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.348 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.358 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.371 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.380 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.387 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.394 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:19.401 UTC [2054] WARNING: there is no transaction in progress
2026-08-08 07:08:20,019 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 07:08:20,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 07:08:20,287 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 07:08:20,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 07:08:20,560 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 07:08:20,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 07:08:20,802 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 07:08:20,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 07:08:21,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 07:08:21,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bw88.c.gcp-upstream.internal';"
2026-08-08 07:08:21,586 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 07:08:21,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 07:08:21,850 [I] DB_PATCHES have already been applied
I20260808 07:08:21.855952 18685 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskERtyUc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 07:08:21.856024 18685 mini_ranger.cc:193] Using host: 127.18.63.84
I20260808 07:08:21.860042 18685 mini_ranger.cc:227] Ranger admin URL: http://127.18.63.84:53263
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 7:08:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 07:08:23.136680 18685 mini_ranger.cc:160] Time spent starting Ranger: real 8.932s user 0.000s sys 0.011s
Aug 08, 2026 7:08:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 7:08:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 08, 2026 7:08:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 7:08:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 7:08:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 07:08:45.389356 18685 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangerkms_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/rangerkms_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/HTTP_127.18.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 07:08:45 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172925, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 07:08:45.493748 18685 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 07:08:45.505928 18685 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 07:08:46.402245 18685 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 07:08:46.402338 18685 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms
I20260808 07:08:46.402359 18685 mini_ranger_kms.cc:227] Using postgres at 127.18.63.84:59415
2026-08-08 07:08:46,587 [I] DB FLAVOR :POSTGRES
2026-08-08 07:08:46,587 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 07:08:46,587 [I] Checking connection
2026-08-08 07:08:46,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 07:08:46,820 [I] connection success
2026-08-08 07:08:46,820 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 07:08:46,820 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 07:08:46,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 07:08:47,051 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 07:08:47,051 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 07:08:47,051 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 07:08:47,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.63.84:59415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 07:08:47,338 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 07:08:47.818194 18685 mini_ranger_kms.cc:310] Created kms service
I20260808 07:08:48.005103 18685 mini_ranger_kms.cc:326] Created kudu user
I20260808 07:08:48.053746 18685 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 07:08:48 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172925, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:48.447336 18685 mini_ranger_kms.cc:384] Added ranger policy
I20260808 07:08:48.447563 18685 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms:/tmp/dist-test-taskERtyUc/build/release/bin/postgresql.jar:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskERtyUc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 07:08:48.447588 18685 mini_ranger_kms.cc:240] Using host: 127.18.63.84
I20260808 07:08:48.452523 18685 mini_ranger_kms.cc:276] Ranger KMS PID: 2302
I20260808 07:08:48.452603 18685 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.63.84:46855
Aug 08, 2026 7:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 7:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 7:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 7:08:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 7:08:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 7:08:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 07:08:51.006518 18685 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.513s user 0.000s sys 0.169s
I20260808 07:08:51.006639 18685 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 07:08:51.006688 18685 mini_ranger_kms.cc:401] 127.18.63.84:46855/kms/v1/keys
Aug 08 07:08:51 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 07:08:51 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
Aug 08 07:08:51 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.63.84@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
Aug 08 07:08:54 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:08:54.200347 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33139
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33139
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:54.286599 2367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:54.286821 2367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:54.286844 2367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:54.288087 2367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:08:54.288134 2367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:54.288148 2367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:54.288161 2367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:08:54.288216 2367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:08:54.289865 2367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.63.84:46855/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33139
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:54.290138 2367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:54.290395 2367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:54.296146 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:54.296195 2374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:54.296147 2377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:54.296165 2367 server_base.cc:1034] running on GCE node
I20260808 07:08:54.296664 2367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:54.296969 2367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:54.298134 2367 hybrid_clock.cc:648] HybridClock initialized: now 1786172934298131 us; error 58 us; skew 500 ppm
Aug 08 07:08:54 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172934, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:54.300717 2367 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:08:54.301172 2367 webserver.cc:460] Webserver started at http://127.18.63.126:43707/ using document root <none> and password file <none>
I20260808 07:08:54.301471 2367 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:54.301534 2367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:54.301694 2367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:54 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172934, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:54.356662 2367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "b2812451614f4325a6b0e68cd6818ddc"
format_stamp: "Formatted at 2026-08-08 07:08:54 on dist-test-slave-bw88"
server_key: "6d20f3e3753992521a20658a9e32cb43"
server_key_iv: "e86a3ff1ab2caf20987f794c32b8bc28"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:54.357183 2367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "b2812451614f4325a6b0e68cd6818ddc"
format_stamp: "Formatted at 2026-08-08 07:08:54 on dist-test-slave-bw88"
server_key: "6d20f3e3753992521a20658a9e32cb43"
server_key_iv: "e86a3ff1ab2caf20987f794c32b8bc28"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:54.359107 2367 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 07:08:54.382298 2386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:54.382623 2367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:08:54.382722 2367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "b2812451614f4325a6b0e68cd6818ddc"
format_stamp: "Formatted at 2026-08-08 07:08:54 on dist-test-slave-bw88"
server_key: "6d20f3e3753992521a20658a9e32cb43"
server_key_iv: "e86a3ff1ab2caf20987f794c32b8bc28"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:54.382833 2367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:54.394833 2367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:54.396422 2367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:54.396698 2367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:54.401132 2367 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33139
I20260808 07:08:54.401206 2456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33139 every 8 connection(s)
I20260808 07:08:54.401587 2367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:08:54.402314 2457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:54.402868 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 2367
I20260808 07:08:54.402968 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:08:54.403180 18685 external_mini_cluster.cc:1428] Setting key 470ad9c95f13b878300a4fa0b418e169
I20260808 07:08:54.405548 2457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Bootstrap starting.
I20260808 07:08:54.406354 2457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:54.406649 2457 log.cc:824] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:54.407522 2457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: No bootstrap required, opened a new log
I20260808 07:08:54.409615 2457 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:54.409783 2457 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:54.409854 2457 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2812451614f4325a6b0e68cd6818ddc, State: Initialized, Role: FOLLOWER
I20260808 07:08:54.410030 2457 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:54.410130 2457 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:08:54.410174 2457 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:08:54.410223 2457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:08:54 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:54.410825 2457 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:54.410948 2457 leader_election.cc:302] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2812451614f4325a6b0e68cd6818ddc; no voters:
I20260808 07:08:54.411139 2457 leader_election.cc:288] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:08:54.411191 2466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:08:54.411339 2466 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 LEADER]: Becoming Leader. State: Replica: b2812451614f4325a6b0e68cd6818ddc, State: Running, Role: LEADER
I20260808 07:08:54.411442 2457 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:08:54.411520 2466 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:54.411885 2467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2812451614f4325a6b0e68cd6818ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } } }
I20260808 07:08:54.411966 2467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:54.412194 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2812451614f4325a6b0e68cd6818ddc. Latest consensus state: current_term: 1 leader_uuid: "b2812451614f4325a6b0e68cd6818ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } } }
I20260808 07:08:54.412320 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:54.412477 2475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:08:54.413239 2475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:08:54.413565 2457 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:08:54.414683 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:54.403749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.18.63.126:33139)
0808 07:08:54.403964 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:54.403975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:54.404009 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:08:54.404574 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:54.404583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:54.404654 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:54.404810 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:54.405471 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:54.406896 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:54.407776 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:54.408185 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:54.408305 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:54.410867 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:54.410871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:54.410871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:54.410882 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:54.412971 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:54.413442 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:54.413445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:54.413445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:54.413543 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:54.413763 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:54.413764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:54.413764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:54.414065 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:54.414273 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:54.414352 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:08:54.414473 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260808 07:08:54.419263 2475 catalog_manager.cc:1347] Generated new cluster ID: e8827a2a69424a24816df97c4de1215f
I20260808 07:08:54.419354 2475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:08:54.428814 2475 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:08:54.429580 2475 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:08:54.437641 2475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Generated new TSK 0
I20260808 07:08:54.437800 2475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:08:55.098491 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:55.209190 2526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:55.209460 2526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:55.209488 2526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:55.210909 2526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:55.210963 2526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:55.211001 2526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:08:55.213194 2526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:55.213459 2526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:55.213713 2526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:55.220399 2532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:55.220412 2535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:55.220567 2526 server_base.cc:1034] running on GCE node
W20260808 07:08:55.220651 2531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:55.220989 2526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:55.221320 2526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:55.222564 2526 hybrid_clock.cc:648] HybridClock initialized: now 1786172935222534 us; error 42 us; skew 500 ppm
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:55.225689 2526 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:08:55.226228 2526 webserver.cc:460] Webserver started at http://127.18.63.65:46813/ using document root <none> and password file <none>
I20260808 07:08:55.226459 2526 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:55.226527 2526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:55.226671 2526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:55.238708 2526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "be6fa4c1f8a64b22aad12817c46cc4a1"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fd21045ca811b742644d4667244f7e6a"
server_key_iv: "89cab80ef19d061a31e8bec9b9c2da40"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.239265 2526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "be6fa4c1f8a64b22aad12817c46cc4a1"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fd21045ca811b742644d4667244f7e6a"
server_key_iv: "89cab80ef19d061a31e8bec9b9c2da40"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.241248 2526 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:08:55.255841 2544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.256017 2526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:08:55.256115 2526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "be6fa4c1f8a64b22aad12817c46cc4a1"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fd21045ca811b742644d4667244f7e6a"
server_key_iv: "89cab80ef19d061a31e8bec9b9c2da40"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.256214 2526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:55.276049 2526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:55.278682 2526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:55.278918 2526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:55.279218 2526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:55.279666 2526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:55.279762 2526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.279860 2526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:55.279937 2526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.288112 2526 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:41725
I20260808 07:08:55.288566 2526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:08:55.290907 2702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:41725 every 8 connection(s)
I20260808 07:08:55.297127 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 2526
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:55.297223 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:08:55.297451 18685 external_mini_cluster.cc:1428] Setting key d70b2e76823b9d684e676c4d0e655440
I20260808 07:08:55.301529 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.290277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:32913 (local address 127.18.63.126:33139)
0808 07:08:55.291407 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:55.291410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:55.291418 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:08:55.291446 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:55.291447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:55.291459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:55.291541 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:55.292343 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.293152 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.294687 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.295000 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.295085 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.297718 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:55.297720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:55.297720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:55.297729 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:55.299877 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.300402 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.300403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.300404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.300475 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.300755 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.300756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.300757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.300876 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:55.300965 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:55.301271 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 07:08:55.301427 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":90,"threads_started":1}
I20260808 07:08:55.301661 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.289433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.65:32913)
0808 07:08:55.290475 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:55.290525 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:55.291210 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:55.291675 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:55.291679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:55.291915 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:55.292202 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.292208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.293261 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.293262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.294583 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.294586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.295120 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.295122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.295263 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.295871 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:55.295881 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:55.297588 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:55.299973 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.299980 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.299983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.300304 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.300555 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.300556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.300557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.300677 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.300956 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:55.300963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:55.301108 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 07:08:55.301233 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 07:08:55.301411 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":595,"threads_started":1}
I20260808 07:08:55.302361 2703 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:55.302459 2703 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:55.302661 2703 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:55.303251 2411 ts_manager.cc:194] Registered new tserver with Master: be6fa4c1f8a64b22aad12817c46cc4a1 (127.18.63.65:41725)
I20260808 07:08:55.304234 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:32913
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:08:55.328800 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:55.458885 2710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:55.459257 2710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:55.459352 2710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:55.461033 2710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:55.461149 2710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:55.461270 2710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:08:55.463493 2710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:55.463752 2710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:55.464058 2710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:55.470655 2724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:55.470698 2726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:55.470703 2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:55.471150 2710 server_base.cc:1034] running on GCE node
I20260808 07:08:55.471354 2710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:55.471594 2710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:55.476567 2710 hybrid_clock.cc:648] HybridClock initialized: now 1786172935476027 us; error 548 us; skew 500 ppm
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:55.479962 2710 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:08:55.480489 2710 webserver.cc:460] Webserver started at http://127.18.63.66:38541/ using document root <none> and password file <none>
I20260808 07:08:55.480793 2710 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:55.480859 2710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:55.481062 2710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:55.499765 2710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "9d2a0849c57e47bda5a789f13eac1029"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fe2497a1343ad6e24897342ef2881e39"
server_key_iv: "50c08fa308b91d9ca4cc8f693d60a862"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.500348 2710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "9d2a0849c57e47bda5a789f13eac1029"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fe2497a1343ad6e24897342ef2881e39"
server_key_iv: "50c08fa308b91d9ca4cc8f693d60a862"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.508402 2710 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260808 07:08:55.527019 2735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.527292 2710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:08:55.527405 2710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "9d2a0849c57e47bda5a789f13eac1029"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fe2497a1343ad6e24897342ef2881e39"
server_key_iv: "50c08fa308b91d9ca4cc8f693d60a862"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.527503 2710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:55.540627 2710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:55.543272 2710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:55.543493 2710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:55.543824 2710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:55.544256 2710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:55.544363 2710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.544438 2710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:55.544502 2710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.555191 2710 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:42597
I20260808 07:08:55.555557 2886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:42597 every 8 connection(s)
I20260808 07:08:55.555771 2710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:08:55.555990 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 2710
I20260808 07:08:55.556098 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:08:55.556347 18685 external_mini_cluster.cc:1428] Setting key d40ebd8b1e10fcc862bd1e04d8a23413
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:55.569466 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.557190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:48185 (local address 127.18.63.126:33139)
0808 07:08:55.557249 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:55.557254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:55.557793 (+ 539us) server_negotiation.cc:407] Connection header received
0808 07:08:55.558557 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:55.558558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:55.558577 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:55.558613 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:55.559606 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.560503 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.562245 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.562575 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.562664 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.565574 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:55.565576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:55.565577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:55.565587 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:55.567383 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.567927 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.567929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.567932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.568064 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.568473 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.568474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.568475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.568639 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:55.568710 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:55.569160 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 07:08:55.569287 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:08:55.570333 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.557197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.66:48185)
0808 07:08:55.557647 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:55.557695 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:55.558415 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:55.558852 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:55.558857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:55.559153 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:55.559467 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.559475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.560643 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.560643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.562131 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.562135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.562755 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.562756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.562902 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.563547 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:55.563558 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:55.565406 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:55.567529 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.567537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.567540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.567817 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.568204 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.568205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.568206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.568366 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.568754 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:55.568762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:55.568957 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 07:08:55.569868 (+ 911us) client_negotiation.cc:240] Negotiation successful
0808 07:08:55.570072 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":95,"threads_started":1}
I20260808 07:08:55.571244 2887 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:55.571355 2887 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:55.571570 2887 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:55.572072 2411 ts_manager.cc:194] Registered new tserver with Master: 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597)
I20260808 07:08:55.572618 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:48185
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:08:55.609193 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:55.782474 2894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:55.782821 2894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:55.782902 2894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:55.784788 2894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:55.784899 2894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:55.784972 2894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:08:55.787595 2894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:55.788043 2894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:55.788370 2894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:08:55.799703 2894 server_base.cc:1034] running on GCE node
W20260808 07:08:55.800180 2903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:55.800961 2899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:55.800885 2900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:55.801506 2894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:55.801820 2894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:55.803013 2894 hybrid_clock.cc:648] HybridClock initialized: now 1786172935802985 us; error 52 us; skew 500 ppm
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:55.808434 2894 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:08:55.809029 2894 webserver.cc:460] Webserver started at http://127.18.63.67:44741/ using document root <none> and password file <none>
I20260808 07:08:55.809357 2894 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:55.809465 2894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:08:55.809654 2894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:55.824563 2894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "f447ffe3393c4f7e8c38eeae4bc2c383"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "378c63dbf6900ee1df4ed5f4b6461302"
server_key_iv: "4898975ad4fd84fd70de08f22a545025"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.826072 2894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "f447ffe3393c4f7e8c38eeae4bc2c383"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "378c63dbf6900ee1df4ed5f4b6461302"
server_key_iv: "4898975ad4fd84fd70de08f22a545025"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.829955 2894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260808 07:08:55.846055 2912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.846396 2894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 07:08:55.846565 2894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "f447ffe3393c4f7e8c38eeae4bc2c383"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "378c63dbf6900ee1df4ed5f4b6461302"
server_key_iv: "4898975ad4fd84fd70de08f22a545025"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:55.846671 2894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:55.868623 2894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:55.871908 2894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:55.872114 2894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:55.872478 2894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:55.873077 2894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:55.873167 2894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.873199 2894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:55.873275 2894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:55.899374 2894 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:42023
I20260808 07:08:55.899569 3068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:42023 every 8 connection(s)
I20260808 07:08:55.900120 2894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:08:55.908705 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 2894
I20260808 07:08:55.908828 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:08:55.909108 18685 external_mini_cluster.cc:1428] Setting key 1da649f1dcba24cbf564ffde9c6c3928
Aug 08 07:08:55 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172935, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:55.920238 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.901304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52757 (local address 127.18.63.126:33139)
0808 07:08:55.901388 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:55.901391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:55.905809 (+ 4418us) server_negotiation.cc:407] Connection header received
0808 07:08:55.907088 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:55.907089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:55.907105 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:55.907202 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:55.908768 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.909829 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.911985 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.912349 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.912424 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.916301 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:55.916303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:55.916304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:55.916312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:55.918434 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.919008 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.919018 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.919018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.919099 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:55.919430 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:55.919432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:55.919432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:55.919603 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:55.919688 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:55.920063 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 07:08:55.920150 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:08:55.920804 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:55.901327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.67:52757)
0808 07:08:55.905557 (+ 4230us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:55.905622 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:55.906937 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:55.907348 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:55.907353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:55.907940 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:55.908557 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.908566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.909965 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.909966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.911869 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:55.911872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:55.912557 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:55.912558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:55.912874 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:55.913654 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:55.913673 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:55.916159 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:55.918568 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.918576 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.918578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.918919 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.919190 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:55.919191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:55.919191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:55.919345 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:55.919689 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:55.919697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:55.919877 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 07:08:55.920014 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 07:08:55.920190 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4126,"thread_start_us":261,"threads_started":1}
I20260808 07:08:55.921916 3069 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:55.922056 3069 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:55.922287 3069 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:55.922864 2411 ts_manager.cc:194] Registered new tserver with Master: f447ffe3393c4f7e8c38eeae4bc2c383 (127.18.63.67:42023)
I20260808 07:08:55.923651 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:52757
I20260808 07:08:55.932818 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:08:55.933637 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 2526
I20260808 07:08:55.991806 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 2710
I20260808 07:08:55.999279 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 2894
I20260808 07:08:56.034039 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 2367
I20260808 07:08:56.048813 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33139
--webserver_interface=127.18.63.126
--webserver_port=43707
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33139
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:08:56.149784 3073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:56.149991 3073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:56.150013 3073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:56.151139 3073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:08:56.151187 3073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:56.151204 3073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:56.151219 3073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:08:56.151233 3073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:08:56.153050 3073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33139
--ranger_config_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=43707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:56.153311 3073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:56.153592 3073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:56.158771 3081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.158771 3082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.158912 3073 server_base.cc:1034] running on GCE node
W20260808 07:08:56.159087 3084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.159413 3073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:56.159719 3073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:56.160952 3073 hybrid_clock.cc:648] HybridClock initialized: now 1786172936160940 us; error 30 us; skew 500 ppm
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:56.163890 3073 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:08:56.164354 3073 webserver.cc:460] Webserver started at http://127.18.63.126:43707/ using document root <none> and password file <none>
I20260808 07:08:56.164597 3073 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:56.164678 3073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:56.228631 3073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:08:56.229586 3097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.229820 3073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:08:56.229936 3073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "b2812451614f4325a6b0e68cd6818ddc"
format_stamp: "Formatted at 2026-08-08 07:08:54 on dist-test-slave-bw88"
server_key: "6d20f3e3753992521a20658a9e32cb43"
server_key_iv: "e86a3ff1ab2caf20987f794c32b8bc28"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:56.230423 3073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:56.247025 3073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:56.248770 3073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:56.248893 3073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:56.253273 3073 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33139
I20260808 07:08:56.253324 3165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33139 every 8 connection(s)
I20260808 07:08:56.253726 3073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:08:56.254988 3166 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:08:56.255709 3166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Bootstrap starting.
I20260808 07:08:56.257164 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3073
I20260808 07:08:56.257598 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:41725
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:33139
--webserver_port=46813
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:08:56.264763 3166 log.cc:824] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:56.266829 3166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:08:56.267163 3166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Bootstrap complete.
I20260808 07:08:56.269716 3166 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:56.269951 3166 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2812451614f4325a6b0e68cd6818ddc, State: Initialized, Role: FOLLOWER
I20260808 07:08:56.270140 3166 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:56.270203 3166 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:08:56.270253 3166 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:08:56.270301 3166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:08:56.271278 3166 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:56.271391 3166 leader_election.cc:302] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2812451614f4325a6b0e68cd6818ddc; no voters:
I20260808 07:08:56.271591 3166 leader_election.cc:288] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:08:56.271636 3170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:08:56.271826 3170 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [term 2 LEADER]: Becoming Leader. State: Replica: b2812451614f4325a6b0e68cd6818ddc, State: Running, Role: LEADER
I20260808 07:08:56.271888 3166 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:08:56.272034 3170 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } }
I20260808 07:08:56.272336 3171 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b2812451614f4325a6b0e68cd6818ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } } }
I20260808 07:08:56.272430 3171 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:56.272698 3178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:08:56.273124 3172 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2812451614f4325a6b0e68cd6818ddc. Latest consensus state: current_term: 2 leader_uuid: "b2812451614f4325a6b0e68cd6818ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2812451614f4325a6b0e68cd6818ddc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33139 } } }
I20260808 07:08:56.273263 3172 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc [sys.catalog]: This master's current role is: LEADER
I20260808 07:08:56.273180 3178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:08:56.273470 3178 catalog_manager.cc:1259] Loaded cluster ID: e8827a2a69424a24816df97c4de1215f
I20260808 07:08:56.273523 3178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:08:56.273569 3166 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 07:08:56.278649 3178 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:08:56.279192 3178 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Loaded TSK: 0
I20260808 07:08:56.279505 3178 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:08:56.375875 3168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:56.376219 3168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:56.376291 3168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:56.378021 3168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:56.378086 3168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:56.378129 3168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:08:56.380326 3168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:41725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=46813
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:56.380789 3168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:56.381099 3168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:56.387619 3215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.387643 3214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.387663 3217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.388221 3168 server_base.cc:1034] running on GCE node
I20260808 07:08:56.388440 3168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:56.388700 3168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:56.389873 3168 hybrid_clock.cc:648] HybridClock initialized: now 1786172936389848 us; error 43 us; skew 500 ppm
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:56.393185 3168 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:08:56.393740 3168 webserver.cc:460] Webserver started at http://127.18.63.65:46813/ using document root <none> and password file <none>
I20260808 07:08:56.394037 3168 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:56.394122 3168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:56.421809 3168 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:08:56.425726 3226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.425940 3168 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 07:08:56.426046 3168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "be6fa4c1f8a64b22aad12817c46cc4a1"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fd21045ca811b742644d4667244f7e6a"
server_key_iv: "89cab80ef19d061a31e8bec9b9c2da40"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:56.426539 3168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:56.436923 3168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:56.439064 3168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:56.439237 3168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:56.439493 3168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:56.439878 3168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:56.439932 3168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.439998 3168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:56.440044 3168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.447710 3168 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:41725
I20260808 07:08:56.448206 3168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:08:56.449543 3369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:41725 every 8 connection(s)
I20260808 07:08:56.456851 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3168
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:56.457204 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:42597
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:33139
--webserver_port=38541
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:08:56.461972 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.450353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.65:49493)
0808 07:08:56.450523 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:56.450575 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:56.451115 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:56.451674 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:56.451678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:56.451883 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:56.452074 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.452078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.453224 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.453226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.454090 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.454093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.454779 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.454780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.454936 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.455610 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:56.455619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:56.457299 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:56.459818 (+ 2519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.459843 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.459845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.460164 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.460491 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.460491 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.460492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.460649 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.461186 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:56.461192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:56.461375 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 07:08:56.461487 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 07:08:56.461667 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260808 07:08:56.462699 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.450471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:49493 (local address 127.18.63.126:33139)
0808 07:08:56.450740 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:56.450748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:56.450786 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:08:56.451422 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:56.451427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:56.451481 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:56.451644 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:56.452206 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.453104 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.454197 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.454643 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.454786 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.457473 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:56.457478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:56.457478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:56.457491 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:56.459716 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.460264 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.460269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.460269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.460400 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.460736 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.460737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.460738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.461055 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:56.461139 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:56.462289 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0808 07:08:56.462428 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":112,"threads_started":1}
I20260808 07:08:56.463730 3370 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:56.463847 3370 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.464063 3370 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.465054 3370 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33139 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:08:56.465131 3370 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.465207 3370 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:56.465694 3370 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.465778 3370 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.466156 3370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:08:56.566944 3378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:56.567306 3378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:56.567404 3378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:56.569088 3378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:56.569211 3378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:56.569293 3378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:08:56.571596 3378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:42597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=38541
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:56.571848 3378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:56.572187 3378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:56.578382 3384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.578577 3383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.578881 3378 server_base.cc:1034] running on GCE node
W20260808 07:08:56.578382 3386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.579162 3378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:56.579443 3378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:56.580628 3378 hybrid_clock.cc:648] HybridClock initialized: now 1786172936580612 us; error 29 us; skew 500 ppm
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:56.583905 3378 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:08:56.584437 3378 webserver.cc:460] Webserver started at http://127.18.63.66:38541/ using document root <none> and password file <none>
I20260808 07:08:56.584789 3378 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:56.584939 3378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:56.605535 3378 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 07:08:56.607288 3394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.607558 3378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 07:08:56.607681 3378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "9d2a0849c57e47bda5a789f13eac1029"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "fe2497a1343ad6e24897342ef2881e39"
server_key_iv: "50c08fa308b91d9ca4cc8f693d60a862"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:56.608079 3378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:56.619220 3378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:56.621676 3378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:56.621840 3378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:56.622092 3378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:56.622437 3378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:56.622483 3378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.622514 3378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:56.622593 3378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.630199 3378 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:42597
I20260808 07:08:56.630700 3378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:08:56.631264 3546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:42597 every 8 connection(s)
I20260808 07:08:56.634752 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3378
I20260808 07:08:56.635123 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.63.84:46855/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:42023
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:33139
--webserver_port=44741
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:56.644629 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.631532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:58261 (local address 127.18.63.126:33139)
0808 07:08:56.631586 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:56.631589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:56.633656 (+ 2067us) server_negotiation.cc:407] Connection header received
0808 07:08:56.634389 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:56.634390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:56.634405 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:56.634453 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:56.635316 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.636125 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.637840 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.638182 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.638244 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.640952 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:56.640953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:56.640954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:56.640962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:56.642853 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.643387 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.643389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.643389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.643453 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.643791 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.643792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.643793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.643941 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:56.643991 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:56.644404 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 07:08:56.644510 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:08:56.645221 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.631547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.66:58261)
0808 07:08:56.633474 (+ 1927us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:56.633528 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:56.634250 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:56.634586 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:56.634591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:56.634858 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:56.635183 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.635191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.636256 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.636257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.637721 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.637725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.638304 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.638306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.638468 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.639104 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:56.639115 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:56.640825 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:56.642956 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.642964 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.642966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.643284 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.643541 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.643541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.643542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.643694 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.644046 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:56.644053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:56.644226 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 07:08:56.644784 (+ 558us) client_negotiation.cc:240] Negotiation successful
0808 07:08:56.644977 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1874,"thread_start_us":99,"threads_started":1}
I20260808 07:08:56.646035 3547 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:56.646139 3547 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.646344 3547 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.647017 3547 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33139 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:08:56.647099 3547 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.647171 3547 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:56.647549 3547 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.647655 3547 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.648016 3547 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:08:56.737200 3550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:08:56.737484 3550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:08:56.737522 3550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:08:56.739226 3550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:08:56.739282 3550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:08:56.739320 3550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:08:56.741504 3550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.63.84:46855/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:42023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=44741
--tserver_master_addrs=127.18.63.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:08:56.741757 3550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:08:56.742079 3550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:08:56.748489 3556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.748497 3555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:08:56.748890 3558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:08:56.749442 3550 server_base.cc:1034] running on GCE node
I20260808 07:08:56.749671 3550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:08:56.749965 3550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:08:56.751169 3550 hybrid_clock.cc:648] HybridClock initialized: now 1786172936751138 us; error 46 us; skew 500 ppm
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:56.754664 3550 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:08:56.755239 3550 webserver.cc:460] Webserver started at http://127.18.63.67:44741/ using document root <none> and password file <none>
I20260808 07:08:56.755515 3550 fs_manager.cc:359] Metadata directory not provided
I20260808 07:08:56.755671 3550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:56.784565 3550 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 07:08:56.785828 3574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.786078 3550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:08:56.786192 3550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "f447ffe3393c4f7e8c38eeae4bc2c383"
format_stamp: "Formatted at 2026-08-08 07:08:55 on dist-test-slave-bw88"
server_key: "378c63dbf6900ee1df4ed5f4b6461302"
server_key_iv: "4898975ad4fd84fd70de08f22a545025"
server_key_version: "kuduclusterkey@0"
I20260808 07:08:56.786625 3550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:08:56.803928 3550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:08:56.806463 3550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:08:56.806648 3550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:08:56.806926 3550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:08:56.807318 3550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:08:56.807372 3550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.807404 3550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:08:56.807482 3550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:08:56.816563 3550 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:42023
I20260808 07:08:56.816623 3724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:42023 every 8 connection(s)
I20260808 07:08:56.817042 3550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:08:56 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172936, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:08:56.823128 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3550
I20260808 07:08:56.826949 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.817970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58747 (local address 127.18.63.126:33139)
0808 07:08:56.818048 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:56.818051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:56.818323 (+ 272us) server_negotiation.cc:407] Connection header received
0808 07:08:56.818865 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:56.818866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:56.818881 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:56.818931 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:56.819543 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.820030 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.821027 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.821253 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.821371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.823549 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:56.823550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:56.823551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:56.823560 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:56.825293 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.825830 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.825831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.825832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.825901 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.826203 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.826204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.826204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.826335 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:56.826405 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:56.826754 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 07:08:56.826836 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:08:56.827172 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.817947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33139 (local address 127.18.63.67:58747)
0808 07:08:56.818202 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:56.818233 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:56.818751 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:56.819070 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:56.819074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:56.819251 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:08:56.819439 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.819443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.820126 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.820127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.820908 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:56.820910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.821397 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.821399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:56.821528 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.822026 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:08:56.822034 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:08:56.823410 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:08:56.825427 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.825434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.825437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.825744 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.825980 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:08:56.825981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:08:56.825981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:08:56.826123 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:08:56.826415 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:08:56.826422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:08:56.826596 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 07:08:56.826726 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 07:08:56.826886 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":102,"threads_started":1}
I20260808 07:08:56.828071 3725 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33139
I20260808 07:08:56.828168 3725 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.828347 3725 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.829497 3725 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33139 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 07:08:56.829602 3725 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.829664 3725 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:56.830093 3725 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:56.830181 3725 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
W20260808 07:08:56.830610 3725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:08:56.831451 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:56.823566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.18.63.126:33139)
0808 07:08:56.823673 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:56.823675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:56.823820 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:08:56.823903 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:56.823904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:56.823916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:56.823992 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:56.824777 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.825553 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.826565 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:56.826869 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:56.826932 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:56.828417 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:56.828418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:56.828419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:56.828426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:56.830235 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.830595 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.830596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.830597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.830652 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:56.830896 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:56.830897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:56.830897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:56.831005 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:56.831222 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:56.831261 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 07:08:56.831339 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":60,"threads_started":1}
Aug 08 07:08:57 dist-test-slave-bw88 krb5kdc[1777](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786172937, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:08:57.466368 3370 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:57.466539 3370 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:57.466962 3118 ts_manager.cc:194] Registered new tserver with Master: be6fa4c1f8a64b22aad12817c46cc4a1 (127.18.63.65:41725)
I20260808 07:08:57.467991 3118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:49493
I20260808 07:08:57.658036 3547 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:57.658236 3547 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:57.658582 3118 ts_manager.cc:194] Registered new tserver with Master: 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597)
I20260808 07:08:57.659142 3118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:58261
Aug 08 07:08:57 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172937, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:57.830801 3725 heartbeater.cc:464] Registering TS with master...
I20260808 07:08:57.830981 3725 heartbeater.cc:511] Master 127.18.63.126:33139 requested a full tablet report, sending...
I20260808 07:08:57.831298 3118 ts_manager.cc:194] Registered new tserver with Master: f447ffe3393c4f7e8c38eeae4bc2c383 (127.18.63.67:42023)
I20260808 07:08:57.831820 3118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:58747
I20260808 07:08:57.838411 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:08:57.851732 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.841653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.18.63.126:33139)
0808 07:08:57.843344 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:57.843346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:57.843357 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:08:57.843382 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:57.843383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:57.843396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:57.843725 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:08:57.844554 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.845396 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.846439 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.846785 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.846863 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.848466 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:08:57.848467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:08:57.848468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:08:57.848476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:08:57.850360 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:57.850784 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:57.850785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:57.850786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:57.850852 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:08:57.851119 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:08:57.851120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:08:57.851121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:08:57.851266 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:08:57.851372 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:57.851528 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 07:08:57.851626 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1667,"thread_start_us":81,"threads_started":1}
I20260808 07:08:57.863333 3118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:08:57.865877 3118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:08:57.875342 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.869139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:42023 (local address 127.0.0.1:34864)
0808 07:08:57.869899 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:57.869941 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:57.870148 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:57.870655 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:57.870658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:57.870661 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:57.870744 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.870746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.872330 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.872331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.873616 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.873619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.874975 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.874976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.875066 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.875079 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:08:57.875199 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 07:08:57.875255 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":160,"threads_started":1}
I20260808 07:08:57.876269 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.869146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42597 (local address 127.0.0.1:32960)
0808 07:08:57.870922 (+ 1776us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:57.870932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:57.871151 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:57.871720 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:57.871722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:57.871724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:57.871781 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.871783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.873408 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.873409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.874642 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.874645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.876023 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.876024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.876098 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.876102 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:57.876151 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 07:08:57.876190 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1740,"thread_start_us":83,"threads_started":1}
I20260808 07:08:57.876789 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.869456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.18.63.66:42597)
0808 07:08:57.870299 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:57.870304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:57.871217 (+ 913us) server_negotiation.cc:407] Connection header received
0808 07:08:57.871259 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:57.871261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:57.871277 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:57.871578 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:57.871883 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.873288 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.874747 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.875843 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.875916 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.875983 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:57.876623 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 07:08:57.876667 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":61,"threads_started":1}
I20260808 07:08:57.877871 3460 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9e30837e147999b7646d9b1ba0045 (DEFAULT_TABLE table=test-table [id=247933ff21ae479fa00b781c1a8044a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:57.878279 3460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9e30837e147999b7646d9b1ba0045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:57.880962 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.869545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.18.63.67:42023)
0808 07:08:57.870280 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:57.870285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:57.870298 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:08:57.870336 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:57.870339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:57.870356 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:57.870630 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:57.870839 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.872237 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.873716 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.874808 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.874883 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.874939 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:57.880803 (+ 5864us) server_negotiation.cc:299] Negotiation successful
0808 07:08:57.880863 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":266,"threads_started":1}
I20260808 07:08:57.881811 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.870059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41725 (local address 127.0.0.1:44410)
0808 07:08:57.875886 (+ 5827us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:57.875897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:57.875937 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:57.876739 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:57.876739 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:57.876741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:57.876772 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.876773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.878136 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.878137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.879259 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:57.879264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.881541 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.881542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:57.881616 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.881620 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:57.881682 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 07:08:57.881729 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5815,"thread_start_us":130,"threads_started":1}
I20260808 07:08:57.881947 3635 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9e30837e147999b7646d9b1ba0045 (DEFAULT_TABLE table=test-table [id=247933ff21ae479fa00b781c1a8044a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:57.882226 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:57.869917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.18.63.65:41725)
0808 07:08:57.870818 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:57.870822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:57.876000 (+ 5178us) server_negotiation.cc:407] Connection header received
0808 07:08:57.876053 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:57.876055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:57.876071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:57.876360 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:57.876867 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.878025 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.879375 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:57.880601 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:57.880680 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:57.880744 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:57.882067 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0808 07:08:57.882128 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":67,"threads_started":1}
I20260808 07:08:57.882422 3635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9e30837e147999b7646d9b1ba0045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:57.882942 3284 tablet_service.cc:1467] Processing CreateTablet for tablet c0d9e30837e147999b7646d9b1ba0045 (DEFAULT_TABLE table=test-table [id=247933ff21ae479fa00b781c1a8044a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:08:57.883224 3284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d9e30837e147999b7646d9b1ba0045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:08:57.887827 3765 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Bootstrap starting.
I20260808 07:08:57.889178 3765 tablet_bootstrap.cc:654] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:57.890060 3764 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Bootstrap starting.
I20260808 07:08:57.890594 3765 log.cc:824] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:57.890769 3764 tablet_bootstrap.cc:654] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:57.891153 3764 log.cc:824] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:57.892025 3765 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: No bootstrap required, opened a new log
I20260808 07:08:57.892118 3765 ts_tablet_manager.cc:1397] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:08:57.892113 3766 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Bootstrap starting.
I20260808 07:08:57.892736 3766 tablet_bootstrap.cc:654] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Neither blocks nor log segments found. Creating new log.
I20260808 07:08:57.893062 3766 log.cc:824] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Log is configured to *not* fsync() on all Append() calls
I20260808 07:08:57.894076 3764 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: No bootstrap required, opened a new log
I20260808 07:08:57.894138 3764 ts_tablet_manager.cc:1397] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 07:08:57.894886 3765 raft_consensus.cc:348] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.895062 3765 raft_consensus.cc:374] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:57.895093 3765 raft_consensus.cc:729] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f447ffe3393c4f7e8c38eeae4bc2c383, State: Initialized, Role: FOLLOWER
I20260808 07:08:57.895388 3765 consensus_queue.cc:260] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.896124 3765 ts_tablet_manager.cc:1428] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:08:57.896294 3725 heartbeater.cc:503] Master 127.18.63.126:33139 was elected leader, sending a full tablet report...
I20260808 07:08:57.896709 3764 raft_consensus.cc:348] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.896857 3764 raft_consensus.cc:374] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:57.896888 3764 raft_consensus.cc:729] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6fa4c1f8a64b22aad12817c46cc4a1, State: Initialized, Role: FOLLOWER
I20260808 07:08:57.896989 3764 consensus_queue.cc:260] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.897217 3764 ts_tablet_manager.cc:1428] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:08:57.897523 3370 heartbeater.cc:503] Master 127.18.63.126:33139 was elected leader, sending a full tablet report...
I20260808 07:08:57.898607 3766 tablet_bootstrap.cc:492] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: No bootstrap required, opened a new log
I20260808 07:08:57.898684 3766 ts_tablet_manager.cc:1397] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260808 07:08:57.900846 3766 raft_consensus.cc:348] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.901006 3766 raft_consensus.cc:374] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:08:57.901036 3766 raft_consensus.cc:729] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d2a0849c57e47bda5a789f13eac1029, State: Initialized, Role: FOLLOWER
I20260808 07:08:57.901142 3766 consensus_queue.cc:260] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:57.901567 3766 ts_tablet_manager.cc:1428] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:08:57.901734 3547 heartbeater.cc:503] Master 127.18.63.126:33139 was elected leader, sending a full tablet report...
W20260808 07:08:57.951122 3371 tablet.cc:2369] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:08:58.040889 3773 raft_consensus.cc:482] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:08:58.041142 3773 raft_consensus.cc:504] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:58.041715 3773 leader_election.cc:288] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597), f447ffe3393c4f7e8c38eeae4bc2c383 (127.18.63.67:42023)
Aug 08 07:08:58 dist-test-slave-bw88 krb5kdc[1777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172937, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for HTTP/127.18.63.84@KRBTEST.COM
I20260808 07:08:58.052021 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.042189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:40345 (local address 127.18.63.66:42597)
0808 07:08:58.042256 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:58.042259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:58.047778 (+ 5519us) server_negotiation.cc:407] Connection header received
0808 07:08:58.047910 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:58.047911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:58.047923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:58.047954 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:58.048374 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.049081 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.050899 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.051525 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.051582 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.051592 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:58.051891 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:08:58.051941 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:08:58.052663 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.041842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42597 (local address 127.18.63.65:40345)
0808 07:08:58.047506 (+ 5664us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.047520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:58.047822 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:58.048026 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:58.048027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:58.048028 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:58.048261 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.048266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.049212 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.049213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.050791 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.050794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.051675 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.051677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.051790 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.051795 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:08:58.052367 (+ 572us) client_negotiation.cc:240] Negotiation successful
0808 07:08:58.052415 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5618,"thread_start_us":225,"threads_started":1}
I20260808 07:08:58.053238 3484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2a0849c57e47bda5a789f13eac1029" is_pre_election: true
I20260808 07:08:58.053525 3484 raft_consensus.cc:2454] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6fa4c1f8a64b22aad12817c46cc4a1 in term 0.
I20260808 07:08:58.053978 3228 leader_election.cc:302] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d2a0849c57e47bda5a789f13eac1029, be6fa4c1f8a64b22aad12817c46cc4a1; no voters:
I20260808 07:08:58.054142 3773 raft_consensus.cc:2781] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:08:58.054189 3773 raft_consensus.cc:482] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:08:58.054212 3773 raft_consensus.cc:3037] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:58.055899 3773 raft_consensus.cc:504] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:58.056044 3773 leader_election.cc:288] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [CANDIDATE]: Term 1 election: Requested vote from peers 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597), f447ffe3393c4f7e8c38eeae4bc2c383 (127.18.63.67:42023)
I20260808 07:08:58.056420 3484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2a0849c57e47bda5a789f13eac1029"
I20260808 07:08:58.056504 3484 raft_consensus.cc:3037] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:58.057837 3484 raft_consensus.cc:2454] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6fa4c1f8a64b22aad12817c46cc4a1 in term 1.
I20260808 07:08:58.058086 3228 leader_election.cc:302] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d2a0849c57e47bda5a789f13eac1029, be6fa4c1f8a64b22aad12817c46cc4a1; no voters:
I20260808 07:08:58.058238 3773 raft_consensus.cc:2781] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:08:58.058624 3773 raft_consensus.cc:686] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [term 1 LEADER]: Becoming Leader. State: Replica: be6fa4c1f8a64b22aad12817c46cc4a1, State: Running, Role: LEADER
I20260808 07:08:58.058744 3773 consensus_queue.cc:237] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:58.059703 3118 catalog_manager.cc:5495] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to be6fa4c1f8a64b22aad12817c46cc4a1 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } health_report { overall_health: HEALTHY } } }
W20260808 07:08:58.069890 3726 tablet.cc:2369] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:08:58.072009 3772 raft_consensus.cc:482] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:08:58.072157 3772 raft_consensus.cc:504] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 } } peers { permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 } } peers { permanent_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 41725 } }
I20260808 07:08:58.072726 3772 leader_election.cc:288] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597), be6fa4c1f8a64b22aad12817c46cc4a1 (127.18.63.65:41725)
I20260808 07:08:58.075960 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.042562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46775 (local address 127.18.63.67:42023)
0808 07:08:58.042641 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:58.042643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:58.058039 (+ 15396us) server_negotiation.cc:407] Connection header received
0808 07:08:58.058154 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:58.058155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:58.058166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:58.058195 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:58.059479 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.060177 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.073986 (+ 13809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.075415 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.075505 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.075516 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:58.075793 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 07:08:58.075854 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:08:58.076359 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.042177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:42023 (local address 127.18.63.65:46775)
0808 07:08:58.057786 (+ 15609us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.057798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:58.058075 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:58.058263 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:58.058264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:58.058265 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:58.059368 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.059374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.060506 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.060507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.071461 (+ 10954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.071468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.075511 (+ 4043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.075513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.075660 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.075667 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 07:08:58.076158 (+ 491us) client_negotiation.cc:240] Negotiation successful
0808 07:08:58.076224 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15566,"thread_start_us":275,"threads_started":1}
I20260808 07:08:58.080107 3659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" is_pre_election: true
I20260808 07:08:58.080238 3659 raft_consensus.cc:2454] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6fa4c1f8a64b22aad12817c46cc4a1 in term 0.
I20260808 07:08:58.080466 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383"
I20260808 07:08:58.080544 3658 raft_consensus.cc:3037] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:08:58.081955 3658 raft_consensus.cc:2454] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6fa4c1f8a64b22aad12817c46cc4a1 in term 1.
I20260808 07:08:58.089175 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.073467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:37461 (local address 127.18.63.65:41725)
0808 07:08:58.073533 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:58.073535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:58.085735 (+ 12200us) server_negotiation.cc:407] Connection header received
0808 07:08:58.085839 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:58.085840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:58.085850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:58.085878 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:58.086081 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.086765 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.088168 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.088778 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.088823 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.088835 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:58.089061 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 07:08:58.089101 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:08:58.089702 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.073394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41725 (local address 127.18.63.67:37461)
0808 07:08:58.085418 (+ 12024us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.085431 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:58.085768 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:58.085937 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:58.085938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:58.085939 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:58.086012 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.086014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.086867 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.086868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.088057 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.088061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.088909 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.088910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.088979 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.088983 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:58.089508 (+ 525us) client_negotiation.cc:240] Negotiation successful
0808 07:08:58.089561 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12001,"thread_start_us":320,"threads_started":1}
I20260808 07:08:58.089941 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.073928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:50303 (local address 127.18.63.66:42597)
0808 07:08:58.073987 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:58.073989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:58.075806 (+ 1817us) server_negotiation.cc:407] Connection header received
0808 07:08:58.075920 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:58.075922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:58.075933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:58.075962 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:08:58.082584 (+ 6622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.083366 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.088728 (+ 5362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.089375 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.089424 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.089432 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:58.089819 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 07:08:58.089868 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:08:58.090574 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.073049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42597 (local address 127.18.63.67:50303)
0808 07:08:58.075582 (+ 2533us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.075594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:08:58.075841 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:08:58.082400 (+ 6559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:08:58.082401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:08:58.082403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:08:58.082470 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.082472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.083470 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.083471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.084615 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:08:58.084619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.089520 (+ 4901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.089521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:08:58.089710 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.089714 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:08:58.090278 (+ 564us) client_negotiation.cc:240] Negotiation successful
0808 07:08:58.090324 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2515,"thread_start_us":169,"threads_started":1}
I20260808 07:08:58.090582 3311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be6fa4c1f8a64b22aad12817c46cc4a1" is_pre_election: true
I20260808 07:08:58.090871 3484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d9e30837e147999b7646d9b1ba0045" candidate_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d2a0849c57e47bda5a789f13eac1029" is_pre_election: true
I20260808 07:08:58.090984 3484 raft_consensus.cc:2379] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f447ffe3393c4f7e8c38eeae4bc2c383 in current term 1: Already voted for candidate be6fa4c1f8a64b22aad12817c46cc4a1 in this term.
I20260808 07:08:58.091225 3576 leader_election.cc:302] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f447ffe3393c4f7e8c38eeae4bc2c383; no voters: 9d2a0849c57e47bda5a789f13eac1029, be6fa4c1f8a64b22aad12817c46cc4a1
I20260808 07:08:58.091430 3772 raft_consensus.cc:2726] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 07:08:58.108362 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.104781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.18.63.65:41725)
0808 07:08:58.104827 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:08:58.104829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:08:58.104838 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:08:58.104881 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:08:58.104882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:08:58.104904 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:08:58.104986 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:08:58.105197 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.105961 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.107109 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:08:58.107393 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:08:58.107477 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:08:58.107705 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:08:58.107849 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:08:58.107938 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:08:58.108252 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 07:08:58.108276 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:08:58.111078 3658 raft_consensus.cc:1261] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 1 FOLLOWER]: Refusing update from remote peer be6fa4c1f8a64b22aad12817c46cc4a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:08:58.111531 3773 consensus_queue.cc:1035] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f447ffe3393c4f7e8c38eeae4bc2c383" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:08:58.114298 3484 raft_consensus.cc:1261] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 1 FOLLOWER]: Refusing update from remote peer be6fa4c1f8a64b22aad12817c46cc4a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:08:58.114761 3773 consensus_queue.cc:1035] T c0d9e30837e147999b7646d9b1ba0045 P be6fa4c1f8a64b22aad12817c46cc4a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d2a0849c57e47bda5a789f13eac1029" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:08:58.119869 3785 mvcc.cc:205] Tried to move back new op lower bound from 7316164354499555328 to 7316164354289197056. Current Snapshot: MvccSnapshot[applied={T|T < 7316164354499555328}]
I20260808 07:08:58.120126 3790 mvcc.cc:205] Tried to move back new op lower bound from 7316164354499555328 to 7316164354289197056. Current Snapshot: MvccSnapshot[applied={T|T < 7316164354499555328}]
I20260808 07:08:58.122896 3787 mvcc.cc:205] Tried to move back new op lower bound from 7316164354499555328 to 7316164354289197056. Current Snapshot: MvccSnapshot[applied={T|T < 7316164354499555328}]
I20260808 07:08:58.126055 3118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:08:58.126152 3118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:08:58.127151 3118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2812451614f4325a6b0e68cd6818ddc: Sending DeleteTablet for 3 replicas of tablet c0d9e30837e147999b7646d9b1ba0045
I20260808 07:08:58.128003 3460 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9e30837e147999b7646d9b1ba0045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:58 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:32960
I20260808 07:08:58.128360 3635 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9e30837e147999b7646d9b1ba0045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:58 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:34864
I20260808 07:08:58.128680 3284 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d9e30837e147999b7646d9b1ba0045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:08:58 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:44410
I20260808 07:08:58.129400 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3168
I20260808 07:08:58.137451 3798 tablet_replica.cc:330] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: stopping tablet replica
I20260808 07:08:58.137575 3798 raft_consensus.cc:2229] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:08:58.137712 3798 raft_consensus.cc:2258] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:08:58.139178 3798 ts_tablet_manager.cc:1905] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:08:58.139357 3800 tablet_replica.cc:330] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: stopping tablet replica
I20260808 07:08:58.139505 3800 raft_consensus.cc:2229] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:08:58.142398 3798 ts_tablet_manager.cc:1918] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:08:58.142549 3798 log.cc:1196] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/c0d9e30837e147999b7646d9b1ba0045
I20260808 07:08:58.142995 3798 ts_tablet_manager.cc:1939] T c0d9e30837e147999b7646d9b1ba0045 P 9d2a0849c57e47bda5a789f13eac1029: Deleting consensus metadata
I20260808 07:08:58.143587 3099 catalog_manager.cc:4841] TS 9d2a0849c57e47bda5a789f13eac1029 (127.18.63.66:42597): tablet c0d9e30837e147999b7646d9b1ba0045 (table test-table [id=247933ff21ae479fa00b781c1a8044a6]) successfully deleted
I20260808 07:08:58.145563 3800 raft_consensus.cc:2258] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:08:58.147702 3800 ts_tablet_manager.cc:1905] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:08:58.148018 3101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:41725 (error 108)
I20260808 07:08:58.148675 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3378
I20260808 07:08:58.149885 3800 ts_tablet_manager.cc:1918] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:08:58.149974 3800 log.cc:1196] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/c0d9e30837e147999b7646d9b1ba0045
I20260808 07:08:58.150315 3800 ts_tablet_manager.cc:1939] T c0d9e30837e147999b7646d9b1ba0045 P f447ffe3393c4f7e8c38eeae4bc2c383: Deleting consensus metadata
I20260808 07:08:58.154944 3101 catalog_manager.cc:4841] TS f447ffe3393c4f7e8c38eeae4bc2c383 (127.18.63.67:42023): tablet c0d9e30837e147999b7646d9b1ba0045 (table test-table [id=247933ff21ae479fa00b781c1a8044a6]) successfully deleted
I20260808 07:08:58.161857 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.161710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41725 (local address 127.0.0.1:44436)
0808 07:08:58.161772 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.161816 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 07:08:58.162132 3101 catalog_manager.cc:4584] TS be6fa4c1f8a64b22aad12817c46cc4a1 (127.18.63.65:41725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0d9e30837e147999b7646d9b1ba0045: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41725: connect: Connection refused (error 111)
I20260808 07:08:58.164045 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3550
I20260808 07:08:58.218089 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.217900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41725 (local address 127.0.0.1:44448)
0808 07:08:58.217989 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.218033 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 07:08:58.218814 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:08:58.218696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:41725 (local address 127.0.0.1:44464)
0808 07:08:58.218750 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:08:58.218780 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:08:58.226120 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3073
I20260808 07:08:58.240207 18685 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 07:08:58.595575 18685 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 07:08:58.595702 18685 mini_ranger.cc:66] Stopping Ranger...
I20260808 07:08:58.950300 18685 mini_ranger.cc:68] Stopped Ranger
2026-08-08 07:08:58.950 UTC [1818] LOG: received smart shutdown request
2026-08-08 07:08:58.955 UTC [1818] LOG: background worker "logical replication launcher" (PID 1829) exited with exit code 1
2026-08-08 07:08:58.956 UTC [1824] LOG: shutting down
2026-08-08 07:08:59.124 UTC [1818] LOG: database system is shut down
2026-08-08T07:08:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50092 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:08:59 dist-test-slave-bw88 krb5kdc[3816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:08:59 dist-test-slave-bw88 krb5kdc[3816](info): set up 2 sockets
Aug 08 07:08:59 dist-test-slave-bw88 krb5kdc[3816](info): commencing operation
krb5kdc: starting...
W20260808 07:09:00.453109 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:09:00.580411 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37635
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:00.663255 3833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:00.663507 3833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:00.663546 3833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:00.664654 3833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:00.664726 3833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:00.664752 3833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:00.664781 3833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:00.664813 3833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:00.666338 3833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:00.666626 3833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:00.666886 3833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:00.671885 3847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:00.671892 3846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:00.671904 3850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:00.672158 3833 server_base.cc:1034] running on GCE node
I20260808 07:09:00.672386 3833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:00.672628 3833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:00.673764 3833 hybrid_clock.cc:648] HybridClock initialized: now 1786172940673739 us; error 41 us; skew 500 ppm
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:00.676265 3833 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:00.676738 3833 webserver.cc:460] Webserver started at http://127.18.63.126:35925/ using document root <none> and password file <none>
I20260808 07:09:00.676961 3833 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:00.677006 3833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:00.677182 3833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:00.678329 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "1a666119818b42fa86c3835fadef3852"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "b7322411777987d2eb55b1521f23a6fd"
server_key_iv: "86d155f0cbe1d58dddddd2e951066be1"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.678709 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "1a666119818b42fa86c3835fadef3852"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "b7322411777987d2eb55b1521f23a6fd"
server_key_iv: "86d155f0cbe1d58dddddd2e951066be1"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.680054 3833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:00.680843 3857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:00.681033 3833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:00.681145 3833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "1a666119818b42fa86c3835fadef3852"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "b7322411777987d2eb55b1521f23a6fd"
server_key_iv: "86d155f0cbe1d58dddddd2e951066be1"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.681233 3833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:00.689992 3833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:00.691743 3833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:00.691900 3833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:00.696307 3833 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:37635
I20260808 07:09:00.696352 3928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:37635 every 8 connection(s)
I20260808 07:09:00.696698 3833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:00.697640 3929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:00.699311 3929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:00.705855 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3833
I20260808 07:09:00.705979 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:09:00.706254 18685 external_mini_cluster.cc:1428] Setting key 9d180e3b5d53adf8c17f9b7835098cd7
I20260808 07:09:00.709575 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.700056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.18.63.126:37635)
0808 07:09:00.700248 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:00.700256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:00.700368 (+ 112us) server_negotiation.cc:407] Connection header received
0808 07:09:00.700846 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:00.700848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:00.700864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:00.700966 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:00.701591 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.702428 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.703296 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.703619 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.703708 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.705741 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:00.705750 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:00.705751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:00.705759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:00.707903 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.708275 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.708278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.708278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.708387 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.708670 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.708671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.708671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.708906 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:00.708971 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:00.709226 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 07:09:00.709324 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260808 07:09:00.709607 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.700056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37635 (local address 127.0.0.1:48102)
0808 07:09:00.700231 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.700272 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:00.700722 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:00.700980 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:00.700982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:00.701249 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:00.701500 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.701505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.702518 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.702519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.703200 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.703202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.703703 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.703704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.703760 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.704241 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:00.704249 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:00.705656 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:00.707999 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.708005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.708007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.708196 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.708455 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.708456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.708456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.708588 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.708978 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:00.708985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:00.709063 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 07:09:00.709167 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 07:09:00.709433 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":91,"thread_start_us":101,"threads_started":1}
I20260808 07:09:00.710355 3929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:00.710747 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.710607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49252)
0808 07:09:00.710664 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.710715 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:09:00.710903 3859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
I20260808 07:09:00.711356 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.711232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49266)
0808 07:09:00.711291 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.711324 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 07:09:00.711522 3929 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45593: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
I20260808 07:09:00.732662 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45593
--webserver_interface=127.18.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:00.760771 3929 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } attempt: 1
I20260808 07:09:00.761368 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.761190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49280)
0808 07:09:00.761262 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.761312 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:09:00.761773 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.761666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49292)
0808 07:09:00.761720 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.761751 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:09:00.761871 3929 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45593: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
W20260808 07:09:00.816910 3938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:00.817199 3938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:00.817240 3938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:00.818439 3938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:00.818485 3938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:00.818498 3938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:00.818511 3938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:00.818524 3938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:00.820001 3938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:00.820227 3938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:00.820406 3938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:00.825471 3946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:00.825479 3945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:00.825479 3950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:00.825681 3938 server_base.cc:1034] running on GCE node
I20260808 07:09:00.826099 3938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:00.826434 3938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:00.827587 3938 hybrid_clock.cc:648] HybridClock initialized: now 1786172940827575 us; error 29 us; skew 500 ppm
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:00.830021 3929 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } attempt: 2
I20260808 07:09:00.830403 3938 init.cc:377] Logged in from keytab as kudu/127.18.63.125@KRBTEST.COM (short username kudu)
I20260808 07:09:00.830509 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.830355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49306)
0808 07:09:00.830423 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.830454 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:09:00.830871 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.830751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49308)
0808 07:09:00.830807 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.830843 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:09:00.830978 3938 webserver.cc:460] Webserver started at http://127.18.63.125:42137/ using document root <none> and password file <none>
W20260808 07:09:00.831019 3929 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45593: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45593: connect: Connection refused (error 111)
I20260808 07:09:00.831213 3938 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:00.831260 3938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:00.831352 3938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:00.832358 3938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data/instance:
uuid: "2d7607ccd4cb449399c5b4908f476892"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "afb4b0fd6577354aa04aaf7ccc8d6485"
server_key_iv: "e703e2e0572b7c67fc4b409355afecae"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.832681 3938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal/instance:
uuid: "2d7607ccd4cb449399c5b4908f476892"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "afb4b0fd6577354aa04aaf7ccc8d6485"
server_key_iv: "e703e2e0572b7c67fc4b409355afecae"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.834043 3938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:09:00.834740 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:00.834923 3938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:09:00.835026 3938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "2d7607ccd4cb449399c5b4908f476892"
format_stamp: "Formatted at 2026-08-08 07:09:00 on dist-test-slave-bw88"
server_key: "afb4b0fd6577354aa04aaf7ccc8d6485"
server_key_iv: "e703e2e0572b7c67fc4b409355afecae"
server_key_version: "encryptionkey@0"
I20260808 07:09:00.835112 3938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:00.863884 3938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:00.866425 3938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:00.866600 3938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:00.871475 3938 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.125:45593
I20260808 07:09:00.871528 4032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.125:45593 every 8 connection(s)
I20260808 07:09:00.871892 3938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
I20260808 07:09:00.872833 4033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:00.874511 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:00.878024 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 3938
I20260808 07:09:00.878153 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-1/wal/instance
I20260808 07:09:00.878381 18685 external_mini_cluster.cc:1428] Setting key 859e9ad74f5d1f608a608556e6a74eaf
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:00.883874 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.875240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37635 (local address 127.0.0.1:48118)
0808 07:09:00.875428 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.875471 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:00.875926 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:00.876185 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:00.876188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:00.876431 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:00.876622 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.876627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.877372 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.877374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.878271 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.878274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.878787 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.878788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.878891 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.879393 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:00.879412 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:00.880779 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:00.882440 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.882448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.882450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.882685 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.882920 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.882920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.882921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.883029 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.883347 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:00.883353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:00.883481 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 07:09:00.883565 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:09:00.883686 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260808 07:09:00.883883 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.875257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.18.63.126:37635)
0808 07:09:00.875333 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:00.875335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:00.875570 (+ 235us) server_negotiation.cc:407] Connection header received
0808 07:09:00.876009 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:00.876010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:00.876022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:00.876086 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:00.876703 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.877252 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.878367 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.878703 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.878781 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.880880 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:00.880882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:00.880883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:00.880891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:00.882336 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.882781 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.882782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.882782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.882845 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.883098 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.883099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.883100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.883250 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:00.883316 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:00.883636 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 07:09:00.883748 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:09:00.884809 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.124@KRBTEST.COM" created.
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:09:00.893940 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.885070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49320)
0808 07:09:00.885120 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.885132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:00.885173 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:00.885510 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:00.885511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:00.885622 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:00.885687 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.885688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.886557 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.886558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.887049 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.887050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.887685 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.887686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.887762 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.888181 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:00.888186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:00.890720 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:00.892849 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.892850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.892850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.893022 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.893256 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.893257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.893257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.893311 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.893700 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:00.893701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:00.893739 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:09:00.893793 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:09:00.893873 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:09:00.894099 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.885160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.18.63.125:45593)
0808 07:09:00.885323 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:00.885328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:00.885374 (+ 46us) server_negotiation.cc:407] Connection header received
0808 07:09:00.885398 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:00.885400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:00.885413 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:00.885508 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:00.885787 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.886465 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.887157 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.887551 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.887685 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.890808 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:00.890818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:00.890818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:00.890827 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:00.892754 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.893109 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.893112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.893112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.893181 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.893412 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.893413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.893414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.893618 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:00.893718 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:00.893831 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 07:09:00.893952 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260808 07:09:00.894711 4033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:00.895074 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.894955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54546)
0808 07:09:00.895006 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.895056 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 07:09:00.895298 3961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
I20260808 07:09:00.895736 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.895651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54562)
0808 07:09:00.895696 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.895721 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:09:00.895884 4033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:34895: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
I20260808 07:09:00.907945 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:34895
--webserver_interface=127.18.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:00.922191 3929 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } attempt: 3
Aug 08 07:09:00 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:09:00.929579 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.922576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49322)
0808 07:09:00.922643 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.922656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:00.922724 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:00.922881 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:00.922882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:00.923048 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:00.923122 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.923124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.923801 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.923802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.924337 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:00.924338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.924723 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.924724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:00.924809 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.925310 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:00.925315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:00.926777 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:00.928553 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.928554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.928554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.928723 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.928932 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:00.928932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:00.928933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:00.928987 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:00.929260 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:00.929260 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:00.929298 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:09:00.929421 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:09:00.929509 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:09:00.929627 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.922660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.18.63.125:45593)
0808 07:09:00.922735 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:00.922737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:00.922749 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:09:00.922782 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:00.922783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:00.922801 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:00.922860 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:00.923213 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.923707 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.924421 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:00.924621 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:00.924714 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:00.926872 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:00.926874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:00.926874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:00.926884 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:00.928469 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.928796 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.928797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.928797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.928865 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:00.929058 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:00.929059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:00.929059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:00.929196 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:00.929266 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:00.929450 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 07:09:00.929536 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:09:00.930238 3929 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:00.930593 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.930491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54566)
0808 07:09:00.930538 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.930573 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:09:00.930933 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.930854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54574)
0808 07:09:00.930895 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.930919 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:09:00.931041 3929 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:34895: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
I20260808 07:09:00.945158 4033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } attempt: 1
I20260808 07:09:00.945623 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.945485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54582)
0808 07:09:00.945548 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.945585 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:09:00.945971 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.945873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54592)
0808 07:09:00.945920 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.945947 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 07:09:00.946069 4033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:34895: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
I20260808 07:09:00.962179 3929 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } attempt: 1
I20260808 07:09:00.962790 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.962625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54594)
0808 07:09:00.962699 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.962750 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:09:00.963199 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:00.963094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54608)
0808 07:09:00.963146 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:00.963176 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:09:00.963335 3929 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:34895: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
W20260808 07:09:00.997072 4043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:00.997323 4043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:00.997383 4043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:00.998584 4043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:00.998649 4043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:00.998673 4043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:00.998693 4043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:00.998721 4043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:01.000293 4043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37635,127.18.63.125:45593,127.18.63.124:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:01.000588 4043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:01.000844 4043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:01.006124 4049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:01.006098 4052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:01.006147 4050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:01.006289 4043 server_base.cc:1034] running on GCE node
I20260808 07:09:01.006551 4043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:01.006912 4043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:01.008041 4043 hybrid_clock.cc:648] HybridClock initialized: now 1786172941008022 us; error 39 us; skew 500 ppm
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172941, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:01.010746 4043 init.cc:377] Logged in from keytab as kudu/127.18.63.124@KRBTEST.COM (short username kudu)
I20260808 07:09:01.011193 4043 webserver.cc:460] Webserver started at http://127.18.63.124:33525/ using document root <none> and password file <none>
I20260808 07:09:01.011456 4043 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:01.011528 4043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:01.011684 4043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:01.012795 4043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data/instance:
uuid: "cae68e321285406aa0273faf95ae5ec4"
format_stamp: "Formatted at 2026-08-08 07:09:01 on dist-test-slave-bw88"
server_key: "057f94246caa40ecbb7c46b331218f68"
server_key_iv: "da4eff3e863e436da64b8de4abaebaaf"
server_key_version: "encryptionkey@0"
I20260808 07:09:01.013167 4043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal/instance:
uuid: "cae68e321285406aa0273faf95ae5ec4"
format_stamp: "Formatted at 2026-08-08 07:09:01 on dist-test-slave-bw88"
server_key: "057f94246caa40ecbb7c46b331218f68"
server_key_iv: "da4eff3e863e436da64b8de4abaebaaf"
server_key_version: "encryptionkey@0"
I20260808 07:09:01.014209 4033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } attempt: 2
I20260808 07:09:01.014637 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.014502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54610)
0808 07:09:01.014567 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.014605 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:09:01.014626 4043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:09:01.015048 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.014904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54624)
0808 07:09:01.014967 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.015009 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:09:01.015182 4033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:34895: Network error: Client connection negotiation failed: client connection to 127.18.63.124:34895: connect: Connection refused (error 111)
I20260808 07:09:01.015523 4060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:01.015690 4043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:09:01.015798 4043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "cae68e321285406aa0273faf95ae5ec4"
format_stamp: "Formatted at 2026-08-08 07:09:01 on dist-test-slave-bw88"
server_key: "057f94246caa40ecbb7c46b331218f68"
server_key_iv: "da4eff3e863e436da64b8de4abaebaaf"
server_key_version: "encryptionkey@0"
I20260808 07:09:01.015884 4043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:01.025936 4043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:01.028295 4043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:01.028462 4043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:01.032600 4043 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.124:34895
I20260808 07:09:01.032639 4141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.124:34895 every 8 connection(s)
I20260808 07:09:01.032996 4043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
I20260808 07:09:01.033528 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4043
I20260808 07:09:01.033654 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786172779312146-18685-0/minicluster-data/master-2/wal/instance
I20260808 07:09:01.033840 18685 external_mini_cluster.cc:1428] Setting key 2f55be0e46806ac691566c991b0ba542
I20260808 07:09:01.034060 4142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:01.036520 4142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:01.038491 3929 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } attempt: 2
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172941, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:09:01.046509 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.037204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.18.63.126:37635)
0808 07:09:01.037245 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.037247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.037460 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:09:01.037534 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.037535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.037545 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.037598 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.038199 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.038810 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.039801 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.040094 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.040177 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.042662 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.042664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.042664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.042671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.044485 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.045033 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.045035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.045035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.045099 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.045739 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.045740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.045741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.045943 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.046032 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.046301 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.046412 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:09:01.046669 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.037158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37635 (local address 127.0.0.1:48130)
0808 07:09:01.037319 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.037391 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:01.037467 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:01.037639 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:01.037642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:01.037988 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:01.038079 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.038081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.038948 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.038949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.039693 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.039696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.040201 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.040202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.040305 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.040822 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:01.040833 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:01.042558 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:01.044568 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.044576 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.044578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.044943 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.045187 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.045188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.045189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.045652 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.046024 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:01.046032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:01.046167 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 07:09:01.046265 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:09:01.046418 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260808 07:09:01.047464 4142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:01.047860 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.038744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54634)
0808 07:09:01.038807 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.038816 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:01.038849 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:01.039965 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:01.039967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:01.040144 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:01.040181 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.040183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.040976 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.040977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.041544 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.041546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.041951 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.041952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.042007 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.042406 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:01.042411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:01.044030 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:01.046301 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.046302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.046302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.046582 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.047207 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.047208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.047208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.047281 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.047522 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:01.047523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:01.047568 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 07:09:01.047660 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:09:01.047760 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 07:09:01.047911 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.038784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.18.63.124:34895)
0808 07:09:01.039840 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.039842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.039849 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:09:01.039867 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.039867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.039878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.039948 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.040272 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.040875 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.041627 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.041861 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.041937 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.044137 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.044139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.044139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.044145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.046208 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.047077 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.047078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.047078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.047147 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.047355 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.047356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.047356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.047467 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.047518 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.047694 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.047789 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":75,"threads_started":1}
I20260808 07:09:01.048133 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.034618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54632 (local address 127.18.63.124:34895)
0808 07:09:01.034812 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.034824 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.034860 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:09:01.035351 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.035360 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.035438 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.035601 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.036370 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.037729 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.038705 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.039223 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.039304 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.042621 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.042630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.042630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.042637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.045206 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.045587 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.045590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.045590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.045681 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.045913 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.045914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.045914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.046193 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.046947 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.047391 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.047471 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260808 07:09:01.049729 3929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: Bootstrap starting.
I20260808 07:09:01.050503 3929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:01.050830 3929 log.cc:824] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: Log is configured to *not* fsync() on all Append() calls
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172941, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:09:01.051915 3929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: No bootstrap required, opened a new log
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:01.054282 3929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.054471 3929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:01.054531 3929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a666119818b42fa86c3835fadef3852, State: Initialized, Role: FOLLOWER
I20260808 07:09:01.054699 3929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.055054 3929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:01.055647 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.047711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45593 (local address 127.0.0.1:49324)
0808 07:09:01.047763 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.047775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:01.047813 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:01.048016 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:01.048017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:01.048182 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:01.048242 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.048244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.049107 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.049108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.049871 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.049873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.050240 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.050241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.050310 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.050714 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:01.050719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:01.052061 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:01.054423 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.054424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.054424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.054655 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.054924 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.054925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.054925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.054993 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.055315 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:01.055316 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:01.055360 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:09:01.055440 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:09:01.055534 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:09:01.055735 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.047818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.18.63.125:45593)
0808 07:09:01.047859 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.047861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.047868 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:09:01.047895 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.047896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.047907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.047963 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.048356 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.049019 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.049937 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.050151 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.050238 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.052162 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.052164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.052164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.052171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.054331 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.054766 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.054767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.054768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.054833 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.055072 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.055073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.055074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.055219 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.055303 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.055486 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.055584 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:09:01.056473 4142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:09:01.057266 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.048887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48140 (local address 127.18.63.126:37635)
0808 07:09:01.048918 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.048920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.048929 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:01.048951 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.048951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.048961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.049012 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.049619 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.050237 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.051188 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.051389 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.051473 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.054034 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.054036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.054036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.054044 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.055739 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.056123 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.056124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.056125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.056183 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.056407 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.056408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.056408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.056530 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.056677 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.057079 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.057153 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:09:01.057595 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.057695 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:09:01.058511 4166 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:09:01.058605 4166 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172941, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:09:01.058064 3833 master_runner.cc:186] Error getting master registration for 127.18.63.125:45593: OK, instance_id { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" instance_seqno: 1786172940869888 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:09:01.061143 3833 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:09:01.064204 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.056716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54642)
0808 07:09:01.056811 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.056821 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:01.056854 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:01.056966 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:01.056966 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:01.057103 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:01.057148 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.057150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.058009 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.058010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.058703 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.058706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.059138 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.059139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.059185 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.059586 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:01.059591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:01.060953 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:01.063100 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.063101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.063101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.063304 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.063511 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.063512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.063512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.063567 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.063852 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:01.063852 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:01.063891 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 07:09:01.063977 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:09:01.064075 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 07:09:01.064216 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.056831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.18.63.124:34895)
0808 07:09:01.056868 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.056869 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.056876 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:09:01.056896 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.056897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.056905 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.056959 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.057227 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.057909 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.058770 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.059007 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.059089 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.061036 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.061037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.061038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.061043 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.063010 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.063380 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.063381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.063381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.063435 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.063642 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.063643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.063643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.063781 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.063859 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.063998 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.064130 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":21}
I20260808 07:09:01.065810 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: Bootstrap starting.
I20260808 07:09:01.066465 4142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:01.066778 4142 log.cc:824] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:01.067598 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.058217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.18.63.125:45593)
0808 07:09:01.058250 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.058252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.058375 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:09:01.058396 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.058396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.058405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.058448 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.059065 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.060062 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.061039 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.061529 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.061611 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.064394 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.064396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.064396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.064403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.066171 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.066569 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.066570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.066570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.066627 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.066827 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.066828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.066829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.066952 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.067188 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.067412 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.067485 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:09:01.068082 4142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: No bootstrap required, opened a new log
I20260808 07:09:01.070055 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:09:01.070425 4142 raft_consensus.cc:348] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.070568 4142 raft_consensus.cc:374] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:01.070634 4142 raft_consensus.cc:729] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae68e321285406aa0273faf95ae5ec4, State: Initialized, Role: FOLLOWER
I20260808 07:09:01.070782 4142 consensus_queue.cc:260] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.071067 4170 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.071146 4142 sys_catalog.cc:564] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 07:09:01.073763 4192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:09:01.073866 4192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:09:01.072870 4170 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.074494 4043 master_runner.cc:186] Error getting master registration for 127.18.63.125:45593: OK, instance_id { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" instance_seqno: 1786172940869888 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:09:01.078027 4043 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:09:01.079559 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.072005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.18.63.125:45593)
0808 07:09:01.072042 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.072044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.072051 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:09:01.072073 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.072074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.072086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.072165 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.072887 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.073587 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.074601 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.074886 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.074946 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.076467 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.076468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.076468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.076475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.078094 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.078495 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.078496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.078497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.078549 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.078738 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.078739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.078739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.078861 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.078951 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.079380 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.079458 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:09:01.083009 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.071993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.18.63.124:34895)
0808 07:09:01.072034 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.072036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.072650 (+ 614us) server_negotiation.cc:407] Connection header received
0808 07:09:01.072759 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.072760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.072771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.072802 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.073636 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.074316 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.075287 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.075593 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.075634 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.079929 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.079930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.079931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.079938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.081716 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.082095 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.082096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.082097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.082155 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.082370 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.082371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.082371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.082515 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.082651 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.082802 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.082886 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:09:01.083602 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.072339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.18.63.126:37635)
0808 07:09:01.072381 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.072383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.076052 (+ 3669us) server_negotiation.cc:407] Connection header received
0808 07:09:01.076165 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.076166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.076178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.076214 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.076865 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.077609 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.078448 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.078756 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.078828 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.080560 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.080561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.080562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.080569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.082368 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.082737 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.082738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.082739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.082797 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.083017 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.083019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.083019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.083157 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.083251 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.083437 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.083515 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:09:01.106367 4033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } attempt: 3
Aug 08 07:09:01 dist-test-slave-bw88 krb5kdc[3816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172940, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:09:01.115190 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.106670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:34895 (local address 127.0.0.1:54666)
0808 07:09:01.106723 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:01.106735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:01.106773 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:01.106931 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:01.106932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:01.107145 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:01.107199 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.107202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.108161 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.108162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.109051 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:01.109054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.109732 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.109734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:01.109813 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.110390 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:01.110396 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:01.112018 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:01.114008 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.114009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.114010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.114249 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.114470 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:01.114471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:01.114472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:01.114543 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:01.114831 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:01.114831 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:01.114882 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:09:01.114962 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:09:01.115065 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:09:01.115190 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:01.106737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.18.63.124:34895)
0808 07:09:01.106793 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:01.106795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:01.106822 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:09:01.106846 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:01.106847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:01.106857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:01.106911 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:01.107285 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.108069 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.109165 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:01.109640 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:01.109715 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:01.112118 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:01.112120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:01.112120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:01.112128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:01.113910 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.114331 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.114332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.114333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.114397 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:01.114616 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:01.114616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:01.114617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:01.114759 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:01.114828 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:01.115006 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 07:09:01.115091 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:09:01.117442 4033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: Bootstrap starting.
I20260808 07:09:01.118275 4033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:01.118603 4033 log.cc:824] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:01.119844 4033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: No bootstrap required, opened a new log
I20260808 07:09:01.122300 4033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.122489 4033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:01.122560 4033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d7607ccd4cb449399c5b4908f476892, State: Initialized, Role: FOLLOWER
I20260808 07:09:01.122709 4033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.123072 4033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 07:09:01.125000 4210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:09:01.125106 4210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:09:01.126720 3938 master_runner.cc:263] No leader master found from master 2d7607ccd4cb449399c5b4908f476892
I20260808 07:09:01.126767 3938 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:09:01.123037 4195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.126931 4195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.186555 4170 raft_consensus.cc:482] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:01.186694 4170 raft_consensus.cc:504] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.186962 4170 leader_election.cc:288] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a666119818b42fa86c3835fadef3852 (127.18.63.126:37635), 2d7607ccd4cb449399c5b4908f476892 (127.18.63.125:45593)
I20260808 07:09:01.187556 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae68e321285406aa0273faf95ae5ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d7607ccd4cb449399c5b4908f476892" is_pre_election: true
I20260808 07:09:01.187745 3997 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae68e321285406aa0273faf95ae5ec4 in term 0.
I20260808 07:09:01.188102 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae68e321285406aa0273faf95ae5ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a666119818b42fa86c3835fadef3852" is_pre_election: true
I20260808 07:09:01.188253 3890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae68e321285406aa0273faf95ae5ec4 in term 0.
I20260808 07:09:01.188647 4067 leader_election.cc:302] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a666119818b42fa86c3835fadef3852, cae68e321285406aa0273faf95ae5ec4; no voters:
I20260808 07:09:01.188997 4170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:09:01.189061 4170 raft_consensus.cc:482] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:09:01.189085 4170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:01.190371 4170 raft_consensus.cc:504] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.190575 4170 leader_election.cc:288] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 1a666119818b42fa86c3835fadef3852 (127.18.63.126:37635), 2d7607ccd4cb449399c5b4908f476892 (127.18.63.125:45593)
I20260808 07:09:01.190941 3890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae68e321285406aa0273faf95ae5ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a666119818b42fa86c3835fadef3852"
I20260808 07:09:01.191043 3890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:01.192090 3890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae68e321285406aa0273faf95ae5ec4 in term 1.
I20260808 07:09:01.192348 4067 leader_election.cc:302] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a666119818b42fa86c3835fadef3852, cae68e321285406aa0273faf95ae5ec4; no voters:
I20260808 07:09:01.192528 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cae68e321285406aa0273faf95ae5ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d7607ccd4cb449399c5b4908f476892"
I20260808 07:09:01.192621 3997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 0 FOLLOWER]: Advancing to term 1
W20260808 07:09:01.192687 3864 tablet.cc:2369] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:09:01.193632 3997 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae68e321285406aa0273faf95ae5ec4 in term 1.
I20260808 07:09:01.194952 4170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:01.195144 4170 raft_consensus.cc:686] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [term 1 LEADER]: Becoming Leader. State: Replica: cae68e321285406aa0273faf95ae5ec4, State: Running, Role: LEADER
I20260808 07:09:01.195237 4170 consensus_queue.cc:237] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } }
I20260808 07:09:01.195642 4170 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae68e321285406aa0273faf95ae5ec4. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.195732 4170 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:01.196113 4215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:01.196355 4215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:01.198736 3997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [term 1 FOLLOWER]: Refusing update from remote peer cae68e321285406aa0273faf95ae5ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:01.199007 3890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [term 1 FOLLOWER]: Refusing update from remote peer cae68e321285406aa0273faf95ae5ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:01.199486 4170 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:01.199653 4170 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:01.201069 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae68e321285406aa0273faf95ae5ec4. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.201157 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.202291 4195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cae68e321285406aa0273faf95ae5ec4. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.202395 4195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.203732 4170 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.203815 4170 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:01.204249 4195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.204332 4195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.204598 4216 mvcc.cc:205] Tried to move back new op lower bound from 7316164367146979328 to 7316164367136280576. Current Snapshot: MvccSnapshot[applied={T|T < 7316164367146979328}]
I20260808 07:09:01.205199 4219 mvcc.cc:205] Tried to move back new op lower bound from 7316164367146979328 to 7316164367136280576. Current Snapshot: MvccSnapshot[applied={T|T < 7316164367146979328}]
I20260808 07:09:01.205839 4215 catalog_manager.cc:1347] Generated new cluster ID: f6a338d202cf44cdb95088d1f21309c4
I20260808 07:09:01.205888 4215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:09:01.214135 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.214246 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:09:01.214648 4212 sys_catalog.cc:455] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cae68e321285406aa0273faf95ae5ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a666119818b42fa86c3835fadef3852" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } peers { permanent_uuid: "2d7607ccd4cb449399c5b4908f476892" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45593 } } peers { permanent_uuid: "cae68e321285406aa0273faf95ae5ec4" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 34895 } } }
I20260808 07:09:01.214744 4212 sys_catalog.cc:458] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:01.215791 4215 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:09:01.216523 4215 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:01.228960 4215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cae68e321285406aa0273faf95ae5ec4: Generated new TSK 0
I20260808 07:09:01.229328 4215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:09:02.059115 4166 catalog_manager.cc:1259] Loaded cluster ID: f6a338d202cf44cdb95088d1f21309c4
I20260808 07:09:02.059206 4166 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: loading cluster ID for follower catalog manager: success
I20260808 07:09:02.060024 4166 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: acquiring CA information for follower catalog manager: success
I20260808 07:09:02.060276 4166 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a666119818b42fa86c3835fadef3852: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:09:02.125617 4210 catalog_manager.cc:1259] Loaded cluster ID: f6a338d202cf44cdb95088d1f21309c4
I20260808 07:09:02.125733 4210 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: loading cluster ID for follower catalog manager: success
I20260808 07:09:02.126556 4210 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: acquiring CA information for follower catalog manager: success
I20260808 07:09:02.126890 4210 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d7607ccd4cb449399c5b4908f476892: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:09:02.280314 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3833
I20260808 07:09:02.293509 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 3938
I20260808 07:09:02.304220 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4043
2026-08-08T07:09:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3066 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:09:02 dist-test-slave-bw88 krb5kdc[4235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:09:02 dist-test-slave-bw88 krb5kdc[4235](info): set up 2 sockets
Aug 08 07:09:02 dist-test-slave-bw88 krb5kdc[4235](info): commencing operation
krb5kdc: starting...
W20260808 07:09:03.543809 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:09:03.673895 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:45043
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:45043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:03.758527 4257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:03.758780 4257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:03.758821 4257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:03.759980 4257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:03.760061 4257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:03.760087 4257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:03.760116 4257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:03.760149 4257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:03.761718 4257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:45043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:45043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:03.762003 4257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:03.762259 4257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:09:03.767424 4257 server_base.cc:1034] running on GCE node
W20260808 07:09:03.767407 4268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:03.767395 4266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:03.767376 4265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:03.767814 4257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:03.768079 4257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:03.769260 4257 hybrid_clock.cc:648] HybridClock initialized: now 1786172943769245 us; error 46 us; skew 500 ppm
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172943, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:03.771960 4257 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:03.772415 4257 webserver.cc:460] Webserver started at http://127.18.63.126:39121/ using document root <none> and password file <none>
I20260808 07:09:03.772645 4257 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:03.772688 4257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:03.772781 4257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:03.773893 4257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "3c659648f6b74141a6585976f248092b"
format_stamp: "Formatted at 2026-08-08 07:09:03 on dist-test-slave-bw88"
server_key: "d772e5ee5fc767d8be3896340f726780"
server_key_iv: "4bf7f4d74868caccaf2cbbba65886baa"
server_key_version: "encryptionkey@0"
I20260808 07:09:03.774220 4257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "3c659648f6b74141a6585976f248092b"
format_stamp: "Formatted at 2026-08-08 07:09:03 on dist-test-slave-bw88"
server_key: "d772e5ee5fc767d8be3896340f726780"
server_key_iv: "4bf7f4d74868caccaf2cbbba65886baa"
server_key_version: "encryptionkey@0"
I20260808 07:09:03.775512 4257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:03.776244 4280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:03.776440 4257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:03.776530 4257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "3c659648f6b74141a6585976f248092b"
format_stamp: "Formatted at 2026-08-08 07:09:03 on dist-test-slave-bw88"
server_key: "d772e5ee5fc767d8be3896340f726780"
server_key_iv: "4bf7f4d74868caccaf2cbbba65886baa"
server_key_version: "encryptionkey@0"
I20260808 07:09:03.776587 4257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:03.798228 4257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:03.800572 4257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:03.800715 4257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:03.804761 4257 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:45043
I20260808 07:09:03.805090 4346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:45043 every 8 connection(s)
I20260808 07:09:03.805159 4257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:03.805930 4350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:03.808283 4350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: Bootstrap starting.
I20260808 07:09:03.808882 4350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:03.809195 4350 log.cc:824] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:03.809831 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4257
I20260808 07:09:03.809978 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:09:03.810210 18685 external_mini_cluster.cc:1428] Setting key fd58cfc475ed4df29412bc1e25584daa
I20260808 07:09:03.810285 4350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: No bootstrap required, opened a new log
I20260808 07:09:03.812009 4350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } }
I20260808 07:09:03.812182 4350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:03.812250 4350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c659648f6b74141a6585976f248092b, State: Initialized, Role: FOLLOWER
I20260808 07:09:03.812389 4350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } }
I20260808 07:09:03.812472 4350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:03.812503 4350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:03.812549 4350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:03.813614 4350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } }
I20260808 07:09:03.813725 4350 leader_election.cc:302] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c659648f6b74141a6585976f248092b; no voters:
I20260808 07:09:03.813895 4350 leader_election.cc:288] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:09:03.813969 4355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:03.814141 4350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:03.814162 4355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [term 1 LEADER]: Becoming Leader. State: Replica: 3c659648f6b74141a6585976f248092b, State: Running, Role: LEADER
I20260808 07:09:03.814396 4355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } }
I20260808 07:09:03.814846 4356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c659648f6b74141a6585976f248092b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } } }
I20260808 07:09:03.814926 4356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:03.815254 4357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c659648f6b74141a6585976f248092b. Latest consensus state: current_term: 1 leader_uuid: "3c659648f6b74141a6585976f248092b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c659648f6b74141a6585976f248092b" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 45043 } } }
I20260808 07:09:03.815330 4357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b [sys.catalog]: This master's current role is: LEADER
W20260808 07:09:03.816155 4373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:09:03.816221 4373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:09:03.816320 4364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:03.816504 4364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:03.818264 4364 catalog_manager.cc:1347] Generated new cluster ID: 3ded08df8efe4c9a891d37f236e42edd
I20260808 07:09:03.818322 4364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:03.823390 4354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:03.810723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.18.63.126:45043)
0808 07:09:03.812304 (+ 1581us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:03.812314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:03.812346 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:09:03.812464 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:03.812471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:03.812530 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:03.812700 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:03.813301 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:03.814734 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:03.815869 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:03.816428 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:03.816538 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:03.819498 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:03.819502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:03.819503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:03.819515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:03.821647 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:03.822040 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:03.822044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:03.822044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:03.822181 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:03.822399 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:03.822400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:03.822400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:03.822787 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:03.822952 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:03.823028 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 07:09:03.823183 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1527,"thread_start_us":75,"threads_started":1}
I20260808 07:09:03.824347 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:09:03.849987 4364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:09:03.850848 4364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:03.862200 4364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c659648f6b74141a6585976f248092b: Generated new TSK 0
I20260808 07:09:03.862447 4364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:09:03.878528 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4257
2026-08-08T07:09:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1569 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4381](info): set up 2 sockets
Aug 08 07:09:03 dist-test-slave-bw88 krb5kdc[4381](info): commencing operation
krb5kdc: starting...
W20260808 07:09:05.099651 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:09:05.228955 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:05.313650 4398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:05.313913 4398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:05.313951 4398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:05.315106 4398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:05.315174 4398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:05.315198 4398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:05.315219 4398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:05.315248 4398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:05.316741 4398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:05.317027 4398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:05.317265 4398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:05.322373 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.322397 4410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.322397 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:05.322414 4398 server_base.cc:1034] running on GCE node
I20260808 07:09:05.322817 4398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:05.323125 4398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:05.324285 4398 hybrid_clock.cc:648] HybridClock initialized: now 1786172945324273 us; error 31 us; skew 500 ppm
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:05.331525 4398 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:05.332099 4398 webserver.cc:460] Webserver started at http://127.18.63.126:37269/ using document root <none> and password file <none>
I20260808 07:09:05.332346 4398 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:05.332428 4398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:05.332590 4398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:05.333822 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "d59ad05803f84285a7ed988a92c309f4"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "e254b0b38f1867e617d246c46104c0c5"
server_key_iv: "5c3746164d4153ab475386155714f547"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.334214 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "d59ad05803f84285a7ed988a92c309f4"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "e254b0b38f1867e617d246c46104c0c5"
server_key_iv: "5c3746164d4153ab475386155714f547"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.335600 4398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:05.336354 4420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.336534 4398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:05.336649 4398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "d59ad05803f84285a7ed988a92c309f4"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "e254b0b38f1867e617d246c46104c0c5"
server_key_iv: "5c3746164d4153ab475386155714f547"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.336726 4398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:05.362370 4398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:05.364868 4398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:05.365052 4398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:05.369379 4398 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33635
I20260808 07:09:05.369423 4499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33635 every 8 connection(s)
I20260808 07:09:05.369833 4398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:05.370872 4500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:05.373219 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap starting.
I20260808 07:09:05.373883 4500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:05.374135 4500 log.cc:824] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:05.374828 4500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: No bootstrap required, opened a new log
I20260808 07:09:05.375385 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4398
I20260808 07:09:05.375509 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:09:05.375737 18685 external_mini_cluster.cc:1428] Setting key c87e9a99a5324dcc3df86cee4b2eeaef
I20260808 07:09:05.377030 4500 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:05.377223 4500 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:05.377269 4500 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Initialized, Role: FOLLOWER
I20260808 07:09:05.377441 4500 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:05.377540 4500 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:05.377574 4500 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:05.377620 4500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:05.378616 4500 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:05.378710 4500 leader_election.cc:302] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59ad05803f84285a7ed988a92c309f4; no voters:
I20260808 07:09:05.378914 4500 leader_election.cc:288] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:09:05.378959 4506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:05.379146 4506 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 LEADER]: Becoming Leader. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Running, Role: LEADER
I20260808 07:09:05.379223 4500 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:05.379268 4506 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:05.379714 4507 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:05.379791 4507 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:05.379936 4508 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59ad05803f84285a7ed988a92c309f4. Latest consensus state: current_term: 1 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:05.379978 4508 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
W20260808 07:09:05.381327 4523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:09:05.381695 4523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:09:05.381781 4514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:05.381953 4514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:05.383524 4514 catalog_manager.cc:1347] Generated new cluster ID: b71c57024afe4d1493d6997c28cfbfa5
I20260808 07:09:05.383584 4514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:09:05.387447 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.376295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.18.63.126:33635)
0808 07:09:05.376517 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.376525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.376552 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:09:05.377010 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.377016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.377063 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.377184 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:05.378055 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.379190 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.380317 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.380824 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.380934 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.383534 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:05.383539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:05.383539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:05.383552 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:05.385578 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.385956 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.385959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.385959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.386085 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.386259 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.386260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.386260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.386543 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:05.386693 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.387106 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.387242 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":123,"threads_started":1}
I20260808 07:09:05.390614 4514 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:09:05.391320 4514 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:05.401669 4514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Generated new TSK 0
I20260808 07:09:05.402004 4514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:09:05.438428 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:05.541962 4529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:05.542198 4529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:05.542243 4529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:05.543424 4529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:05.543488 4529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:05.543534 4529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:09:05.545632 4529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:05.545877 4529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:05.546197 4529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:05.551652 4536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.551667 4537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.551681 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:05.551931 4529 server_base.cc:1034] running on GCE node
I20260808 07:09:05.552143 4529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:05.552345 4529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:05.553483 4529 hybrid_clock.cc:648] HybridClock initialized: now 1786172945553467 us; error 29 us; skew 500 ppm
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:05.556025 4529 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:09:05.556432 4529 webserver.cc:460] Webserver started at http://127.18.63.65:40185/ using document root <none> and password file <none>
I20260808 07:09:05.556624 4529 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:05.556665 4529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:05.556758 4529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:05.557868 4529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "829e35f1d347ecc5e14ceb3a66b457cf"
server_key_iv: "1488830c773d5a74aba5c951566c7298"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.558193 4529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "829e35f1d347ecc5e14ceb3a66b457cf"
server_key_iv: "1488830c773d5a74aba5c951566c7298"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.559484 4529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:05.560179 4551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.560381 4529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:05.560468 4529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "829e35f1d347ecc5e14ceb3a66b457cf"
server_key_iv: "1488830c773d5a74aba5c951566c7298"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.560525 4529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:05.580586 4529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:05.583163 4529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:05.583284 4529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:05.583492 4529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:05.583842 4529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:05.583880 4529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.583904 4529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:05.583953 4529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.589977 4529 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46193
I20260808 07:09:05.590066 4718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46193 every 8 connection(s)
I20260808 07:09:05.590405 4529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:09:05.595924 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4529
I20260808 07:09:05.596066 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:09:05.596313 18685 external_mini_cluster.cc:1428] Setting key a8b41fdbf96dc6efcb66c1104c9e7de5
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:05.600593 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.591292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:45247 (local address 127.18.63.126:33635)
0808 07:09:05.591353 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.591355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.591579 (+ 224us) server_negotiation.cc:407] Connection header received
0808 07:09:05.592095 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.592097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.592110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.592176 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:05.593152 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.593732 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.594825 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.595035 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.595092 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.597617 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:05.597619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:05.597619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:05.597637 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:05.599091 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.599598 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.599600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.599600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.599678 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.599927 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.599928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.599928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.600079 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:05.600153 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.600413 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.600517 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:09:05.600695 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.591257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:45247)
0808 07:09:05.591446 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.591492 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.592006 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.592259 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.592262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.592857 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:05.593056 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.593061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.593818 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.593819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.594746 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.594749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.595112 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.595113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.595215 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.595755 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:05.595764 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:05.597531 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:05.599217 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.599227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.599229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.599521 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.599750 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.599750 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.599751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.599855 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.600152 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:05.600157 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:05.600277 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.600360 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.600486 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260808 07:09:05.601220 4720 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:05.601336 4720 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:05.601544 4720 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:05.602169 4445 ts_manager.cc:194] Registered new tserver with Master: 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193)
I20260808 07:09:05.603185 4445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:45247
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:09:05.627795 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:05.712983 4726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:05.713192 4726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:05.713217 4726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:05.714387 4726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:05.714432 4726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:05.714489 4726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:09:05.715994 4726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:05.716245 4726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:05.716485 4726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:05.721624 4735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.721657 4738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.721663 4734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:05.722005 4726 server_base.cc:1034] running on GCE node
I20260808 07:09:05.722254 4726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:05.722484 4726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:05.723644 4726 hybrid_clock.cc:648] HybridClock initialized: now 1786172945723619 us; error 48 us; skew 500 ppm
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:05.726207 4726 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:09:05.726655 4726 webserver.cc:460] Webserver started at http://127.18.63.66:41179/ using document root <none> and password file <none>
I20260808 07:09:05.726863 4726 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:05.726904 4726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:05.726994 4726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:05.727972 4726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "208a7a6fcdfa8df36e1f2d1bad72413a"
server_key_iv: "c9c9d39e0c0b672f495bedcf23fca760"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.728264 4726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "208a7a6fcdfa8df36e1f2d1bad72413a"
server_key_iv: "c9c9d39e0c0b672f495bedcf23fca760"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.729558 4726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:09:05.730234 4749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.730420 4726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.730527 4726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "208a7a6fcdfa8df36e1f2d1bad72413a"
server_key_iv: "c9c9d39e0c0b672f495bedcf23fca760"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.730600 4726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:05.754908 4726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:05.757491 4726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:05.757671 4726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:05.757936 4726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:05.758258 4726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:05.758315 4726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.758360 4726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:05.758386 4726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.765043 4726 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:37229
I20260808 07:09:05.765101 4902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:37229 every 8 connection(s)
I20260808 07:09:05.765491 4726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:05.774139 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4726
I20260808 07:09:05.774266 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:09:05.774497 18685 external_mini_cluster.cc:1428] Setting key 0aa05045e7d0a7d94435073187586b10
I20260808 07:09:05.774878 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.766285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:40709 (local address 127.18.63.126:33635)
0808 07:09:05.766353 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.766355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.766552 (+ 197us) server_negotiation.cc:407] Connection header received
0808 07:09:05.767057 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.767058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.767067 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.767142 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:05.768081 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.768617 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.769613 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.769802 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.769857 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.772094 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:05.772096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:05.772096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:05.772103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:05.773393 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.773791 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.773793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.773793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.773849 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.774134 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.774135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.774136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.774298 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:05.774372 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.774690 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.774785 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:09:05.775082 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.766254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:40709)
0808 07:09:05.766417 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.766461 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.766970 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.767202 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.767206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.767783 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:05.767989 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.767994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.768706 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.768707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.769542 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.769544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.769879 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.769879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.769987 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.770555 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:05.770564 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:05.772002 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:05.773478 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.773485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.773487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.773712 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.773959 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.773960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.773960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.774058 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.774380 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:05.774387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:05.774517 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.774653 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.774822 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260808 07:09:05.775590 4903 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:05.775700 4903 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:05.775906 4903 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:05.776296 4445 ts_manager.cc:194] Registered new tserver with Master: b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
I20260808 07:09:05.776794 4445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:40709
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:09:05.800916 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:05.885753 4909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:05.885993 4909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:05.886035 4909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:05.887216 4909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:05.887293 4909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:05.887337 4909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:09:05.888830 4909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:05.889102 4909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:05.889374 4909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:05.894644 4920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:05.894678 4909 server_base.cc:1034] running on GCE node
W20260808 07:09:05.894622 4916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:05.894641 4917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:05.895062 4909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:05.895354 4909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:05.896523 4909 hybrid_clock.cc:648] HybridClock initialized: now 1786172945896505 us; error 40 us; skew 500 ppm
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:05.899137 4909 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:09:05.899658 4909 webserver.cc:460] Webserver started at http://127.18.63.67:42567/ using document root <none> and password file <none>
I20260808 07:09:05.899924 4909 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:05.899994 4909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:05.900151 4909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:05.901211 4909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "13684f39a48943529863a9a727f034d5"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "5778ea792251d32f42cc4e5065770d1b"
server_key_iv: "f34957751289e1ffc528e43016d0052f"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.901638 4909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "13684f39a48943529863a9a727f034d5"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "5778ea792251d32f42cc4e5065770d1b"
server_key_iv: "f34957751289e1ffc528e43016d0052f"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.902990 4909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:05.903753 4930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.903927 4909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:05.904038 4909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "13684f39a48943529863a9a727f034d5"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "5778ea792251d32f42cc4e5065770d1b"
server_key_iv: "f34957751289e1ffc528e43016d0052f"
server_key_version: "encryptionkey@0"
I20260808 07:09:05.904114 4909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:05.917812 4909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:05.919845 4909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:05.920014 4909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:05.920270 4909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:05.920604 4909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:05.920663 4909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.920710 4909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:05.920744 4909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:05.927279 4909 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46195
I20260808 07:09:05.927327 5083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46195 every 8 connection(s)
I20260808 07:09:05.927654 4909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:09:05 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:05.936713 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 4909
I20260808 07:09:05.936807 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:09:05.936879 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.928414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:42117 (local address 127.18.63.126:33635)
0808 07:09:05.928498 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.928501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.928675 (+ 174us) server_negotiation.cc:407] Connection header received
0808 07:09:05.929174 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.929175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.929186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.929240 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:05.930274 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.930768 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.931740 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.931920 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.931972 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.934170 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:05.934172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:05.934172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:05.934179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:05.935510 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.935905 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.935906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.935906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.935984 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.936216 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.936217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.936217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.936334 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:05.936391 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.936673 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.936762 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:09:05.936985 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.928367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:42117)
0808 07:09:05.928551 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.928588 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.929077 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.929293 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.929296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.929980 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:05.930190 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.930196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.930851 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.930851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.931663 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.931665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.931995 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.931996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.932100 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.932583 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:05.932592 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:05.934071 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:05.935603 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.935611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.935612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.935832 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.936049 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:05.936049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:05.936050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:05.936156 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:05.936392 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:05.936396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:05.936513 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.936596 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.936720 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260808 07:09:05.937005 18685 external_mini_cluster.cc:1428] Setting key 7d52c053087bf90568e6647a4f5d2731
I20260808 07:09:05.937609 5084 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:05.937732 5084 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:05.937928 5084 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:05.938337 4445 ts_manager.cc:194] Registered new tserver with Master: 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195)
I20260808 07:09:05.938688 4445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:42117
I20260808 07:09:05.949012 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:09:05.956542 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.950468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.18.63.126:33635)
0808 07:09:05.950532 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.950534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.950657 (+ 123us) server_negotiation.cc:407] Connection header received
0808 07:09:05.950736 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.950737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.950748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.950817 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:05.951384 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.951857 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.952622 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.952819 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.952894 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.954193 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:05.954194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:05.954195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:05.954202 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:05.955312 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.955682 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.955683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.955684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.955746 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:05.955955 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:05.955957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:05.955957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:05.956119 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:05.956230 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.956351 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.956436 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:09:05.958132 4445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:09:05.959692 4445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:09:05.966746 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:37229 (local address 127.0.0.1:49458)
0808 07:09:05.962823 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.962834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.962885 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.963314 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.963315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.963317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:05.963350 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.963351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964428 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.964429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.965749 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.965751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966440 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.966520 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966534 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.966603 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.966657 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":87,"thread_start_us":93,"threads_started":1}
I20260808 07:09:05.966849 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.18.63.67:46195)
0808 07:09:05.962590 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.962595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.962605 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:09:05.962657 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.962659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.962676 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.962762 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:05.963070 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964245 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.965838 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966520 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966589 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966656 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.966727 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.966770 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":118,"threads_started":1}
I20260808 07:09:05.966957 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.18.63.65:46193)
0808 07:09:05.962518 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.962524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.962861 (+ 337us) server_negotiation.cc:407] Connection header received
0808 07:09:05.963020 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.963023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.963042 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.963125 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:05.963334 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964417 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.965667 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966396 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966470 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966527 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.966819 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.966865 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260808 07:09:05.967283 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.18.63.66:37229)
0808 07:09:05.963084 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:05.963088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:05.963097 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:05.963147 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:05.963149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:05.963162 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:05.963234 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:05.963414 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964285 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.965852 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966349 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966440 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966503 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:05.967184 (+ 681us) server_negotiation.cc:299] Negotiation successful
0808 07:09:05.967224 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":88,"threads_started":1}
I20260808 07:09:05.966871 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46195 (local address 127.0.0.1:49124)
0808 07:09:05.962346 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.962389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.962472 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.962868 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.962871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.962874 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:05.962962 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.962965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964355 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.964356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.965748 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.965751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966591 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.966654 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966658 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.966704 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.966741 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":37,"thread_start_us":54,"threads_started":1}
I20260808 07:09:05.966936 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:05.962186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46193 (local address 127.0.0.1:34468)
0808 07:09:05.962794 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:05.962804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:05.962887 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:05.963220 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:05.963221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:05.963223 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:05.963267 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.963268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.964515 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.964516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.965566 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:05.965569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:05.966566 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:05.966567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:05.966629 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:05.966633 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:05.966685 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 07:09:05.966734 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":102,"threads_started":1}
I20260808 07:09:05.968189 4989 tablet_service.cc:1467] Processing CreateTablet for tablet 73b692970b2141f99344a0cf6e2de589 (DEFAULT_TABLE table=test-table [id=9210a2a46f7b4315877be601a1b9d421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:05.968267 4809 tablet_service.cc:1467] Processing CreateTablet for tablet 73b692970b2141f99344a0cf6e2de589 (DEFAULT_TABLE table=test-table [id=9210a2a46f7b4315877be601a1b9d421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:05.968307 4629 tablet_service.cc:1467] Processing CreateTablet for tablet 73b692970b2141f99344a0cf6e2de589 (DEFAULT_TABLE table=test-table [id=9210a2a46f7b4315877be601a1b9d421]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:05.968588 4989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b692970b2141f99344a0cf6e2de589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:05.968600 4809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b692970b2141f99344a0cf6e2de589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:05.968624 4629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b692970b2141f99344a0cf6e2de589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:05.971313 5109 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Bootstrap starting.
I20260808 07:09:05.972064 5109 tablet_bootstrap.cc:654] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:05.972072 5110 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap starting.
I20260808 07:09:05.972820 5109 log.cc:824] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:05.972870 5110 tablet_bootstrap.cc:654] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:05.973469 5110 log.cc:824] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:05.973526 5111 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Bootstrap starting.
I20260808 07:09:05.974184 5111 tablet_bootstrap.cc:654] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:05.974535 5110 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: No bootstrap required, opened a new log
I20260808 07:09:05.974627 5110 ts_tablet_manager.cc:1397] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 07:09:05.974774 5111 log.cc:824] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:05.975334 5109 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: No bootstrap required, opened a new log
I20260808 07:09:05.975473 5109 ts_tablet_manager.cc:1397] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:09:05.975725 5111 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: No bootstrap required, opened a new log
I20260808 07:09:05.975808 5111 ts_tablet_manager.cc:1397] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:09:05.976881 5110 raft_consensus.cc:348] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.977072 5110 raft_consensus.cc:374] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:05.977110 5110 raft_consensus.cc:729] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e07e1fcdf84c278fe56050c5e2d53c, State: Initialized, Role: FOLLOWER
I20260808 07:09:05.977293 5110 consensus_queue.cc:260] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.977239 5111 raft_consensus.cc:348] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.977466 5111 raft_consensus.cc:374] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:05.977537 5111 raft_consensus.cc:729] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13684f39a48943529863a9a727f034d5, State: Initialized, Role: FOLLOWER
I20260808 07:09:05.977675 5110 ts_tablet_manager.cc:1428] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:09:05.977702 5111 consensus_queue.cc:260] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.977854 4720 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
I20260808 07:09:05.978004 5111 ts_tablet_manager.cc:1428] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:09:05.978034 5084 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
I20260808 07:09:05.978755 5109 raft_consensus.cc:348] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.978931 5109 raft_consensus.cc:374] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:05.979010 5109 raft_consensus.cc:729] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41a27da1ced4afc841f9c3c86ea88e1, State: Initialized, Role: FOLLOWER
I20260808 07:09:05.979202 5109 consensus_queue.cc:260] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:05.979493 5109 ts_tablet_manager.cc:1428] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 07:09:05.979555 4903 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
W20260808 07:09:06.016127 4904 tablet.cc:2369] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:09:06.091238 4721 tablet.cc:2369] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:09:06.127867 5115 raft_consensus.cc:482] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:06.128008 5115 raft_consensus.cc:504] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:06.128469 5115 leader_election.cc:288] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
I20260808 07:09:06.131603 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.128847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:41727 (local address 127.18.63.66:37229)
0808 07:09:06.128906 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.128909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.129004 (+ 95us) server_negotiation.cc:407] Connection header received
0808 07:09:06.129082 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.129088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.129101 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.129169 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:06.129357 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.129842 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.130839 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.131243 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.131329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.131338 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.131493 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.131533 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:09:06.131628 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.128707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:37229 (local address 127.18.63.65:41727)
0808 07:09:06.128918 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.128930 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.129006 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.129175 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.129176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.129178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:06.129261 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.129263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.129926 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.129927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.130746 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.130749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.131329 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.131330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.131416 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.131421 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.131488 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.131530 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260808 07:09:06.131695 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.128903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46525 (local address 127.18.63.67:46195)
0808 07:09:06.128972 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.128986 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.129049 (+ 63us) server_negotiation.cc:407] Connection header received
0808 07:09:06.129130 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.129131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.129143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.129209 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:06.129453 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.129919 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.130922 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.131313 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.131401 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.131410 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.131581 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.131619 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:09:06.131734 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.128800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46195 (local address 127.18.63.65:46525)
0808 07:09:06.128958 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.128970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.129061 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.129225 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.129226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.129228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:06.129381 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.129383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.130038 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.130039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.130829 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.130831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.131402 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.131403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.131493 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.131496 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.131572 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.131615 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":16,"thread_start_us":88,"threads_started":1}
I20260808 07:09:06.132090 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b692970b2141f99344a0cf6e2de589" candidate_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13684f39a48943529863a9a727f034d5" is_pre_election: true
I20260808 07:09:06.132104 4834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b692970b2141f99344a0cf6e2de589" candidate_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" is_pre_election: true
I20260808 07:09:06.132279 5015 raft_consensus.cc:2454] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e07e1fcdf84c278fe56050c5e2d53c in term 0.
I20260808 07:09:06.132301 4834 raft_consensus.cc:2454] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e07e1fcdf84c278fe56050c5e2d53c in term 0.
I20260808 07:09:06.132522 4556 leader_election.cc:302] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13684f39a48943529863a9a727f034d5, 58e07e1fcdf84c278fe56050c5e2d53c; no voters:
I20260808 07:09:06.132690 5115 raft_consensus.cc:2781] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:09:06.132771 5115 raft_consensus.cc:482] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:09:06.132803 5115 raft_consensus.cc:3037] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:06.133734 5115 raft_consensus.cc:504] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:06.133891 5115 leader_election.cc:288] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 1 election: Requested vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
I20260808 07:09:06.134070 4834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b692970b2141f99344a0cf6e2de589" candidate_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
I20260808 07:09:06.134176 4834 raft_consensus.cc:3037] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:06.134183 5015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b692970b2141f99344a0cf6e2de589" candidate_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13684f39a48943529863a9a727f034d5"
I20260808 07:09:06.134331 5015 raft_consensus.cc:3037] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:06.135015 4834 raft_consensus.cc:2454] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e07e1fcdf84c278fe56050c5e2d53c in term 1.
I20260808 07:09:06.135164 5015 raft_consensus.cc:2454] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e07e1fcdf84c278fe56050c5e2d53c in term 1.
I20260808 07:09:06.135283 4554 leader_election.cc:302] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c, b41a27da1ced4afc841f9c3c86ea88e1; no voters:
I20260808 07:09:06.135380 5115 raft_consensus.cc:2781] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:06.135531 5115 raft_consensus.cc:686] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 LEADER]: Becoming Leader. State: Replica: 58e07e1fcdf84c278fe56050c5e2d53c, State: Running, Role: LEADER
I20260808 07:09:06.135679 5115 consensus_queue.cc:237] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:06.136582 4445 catalog_manager.cc:5495] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c reported cstate change: term changed from 0 to 1, leader changed from <none> to 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65). New cstate: current_term: 1 leader_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } health_report { overall_health: UNKNOWN } } }
I20260808 07:09:06.148970 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.146574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.18.63.65:46193)
0808 07:09:06.146648 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.146650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.146659 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:06.146693 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.146694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.146719 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.146795 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:06.146938 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.147393 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.148170 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.148360 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.148436 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.148622 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:06.148719 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:09:06.148824 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.148870 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.148891 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:09:06.150843 5015 raft_consensus.cc:1261] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Refusing update from remote peer 58e07e1fcdf84c278fe56050c5e2d53c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:06.150861 4834 raft_consensus.cc:1261] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Refusing update from remote peer 58e07e1fcdf84c278fe56050c5e2d53c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:06.151254 5120 consensus_queue.cc:1035] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:06.151386 5115 consensus_queue.cc:1035] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [LEADER]: Connected to new peer: Peer: permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:06.153707 5123 mvcc.cc:205] Tried to move back new op lower bound from 7316164387431415808 to 7316164387372593152. Current Snapshot: MvccSnapshot[applied={T|T < 7316164387431415808}]
I20260808 07:09:06.154023 5125 mvcc.cc:205] Tried to move back new op lower bound from 7316164387431415808 to 7316164387372593152. Current Snapshot: MvccSnapshot[applied={T|T < 7316164387431415808}]
I20260808 07:09:06.156016 4444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:06.156117 4444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:06.156894 4444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Sending DeleteTablet for 3 replicas of tablet 73b692970b2141f99344a0cf6e2de589
I20260808 07:09:06.157299 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4529
I20260808 07:09:06.157272 4989 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b692970b2141f99344a0cf6e2de589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:06 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49124
I20260808 07:09:06.157308 4809 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b692970b2141f99344a0cf6e2de589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:06 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49458
I20260808 07:09:06.162369 5134 tablet_replica.cc:330] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: stopping tablet replica
I20260808 07:09:06.162489 5134 raft_consensus.cc:2229] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:06.162598 5134 raft_consensus.cc:2258] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:06.162797 5135 tablet_replica.cc:330] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: stopping tablet replica
I20260808 07:09:06.162896 5135 raft_consensus.cc:2229] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:06.162990 5135 raft_consensus.cc:2258] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:06.163098 5134 ts_tablet_manager.cc:1905] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:06.163447 5135 ts_tablet_manager.cc:1905] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:06.164609 5134 ts_tablet_manager.cc:1918] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:06.164680 5134 log.cc:1196] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/73b692970b2141f99344a0cf6e2de589
I20260808 07:09:06.164883 5134 ts_tablet_manager.cc:1939] T 73b692970b2141f99344a0cf6e2de589 P 13684f39a48943529863a9a727f034d5: Deleting consensus metadata
I20260808 07:09:06.165256 4425 catalog_manager.cc:4841] TS 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): tablet 73b692970b2141f99344a0cf6e2de589 (table test-table [id=9210a2a46f7b4315877be601a1b9d421]) successfully deleted
I20260808 07:09:06.165650 5135 ts_tablet_manager.cc:1918] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:06.165733 5135 log.cc:1196] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/73b692970b2141f99344a0cf6e2de589
I20260808 07:09:06.165994 5135 ts_tablet_manager.cc:1939] T 73b692970b2141f99344a0cf6e2de589 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting consensus metadata
I20260808 07:09:06.166420 4422 catalog_manager.cc:4841] TS b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): tablet 73b692970b2141f99344a0cf6e2de589 (table test-table [id=9210a2a46f7b4315877be601a1b9d421]) successfully deleted
W20260808 07:09:06.170558 4421 connection.cc:535] client connection to 127.18.63.65:46193 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:09:06.170652 4421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:09:06.171146 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4726
I20260808 07:09:06.171162 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.171014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46193 (local address 127.0.0.1:34478)
0808 07:09:06.171073 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.171133 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:09:06.171371 4421 catalog_manager.cc:4584] TS 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73b692970b2141f99344a0cf6e2de589: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46193: connect: Connection refused (error 111)
I20260808 07:09:06.182437 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4909
I20260808 07:09:06.201481 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 4398
I20260808 07:09:06.212759 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--webserver_interface=127.18.63.126
--webserver_port=37269
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:06.298785 5138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:06.299036 5138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:06.299077 5138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:06.300235 5138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:06.300304 5138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:06.300333 5138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:06.300364 5138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:06.300390 5138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:06.302013 5138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=37269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:06.302312 5138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:06.302547 5138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:06.312626 5150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.312614 5147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.312613 5145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:06.312721 5138 server_base.cc:1034] running on GCE node
I20260808 07:09:06.313191 5138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:06.313627 5138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:06.314808 5138 hybrid_clock.cc:648] HybridClock initialized: now 1786172946314793 us; error 47 us; skew 500 ppm
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:06.317540 5138 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:06.318035 5138 webserver.cc:460] Webserver started at http://127.18.63.126:37269/ using document root <none> and password file <none>
I20260808 07:09:06.318308 5138 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:06.318370 5138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:06.319729 5138 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:06.320379 5159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.320568 5138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:06.320676 5138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "d59ad05803f84285a7ed988a92c309f4"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "e254b0b38f1867e617d246c46104c0c5"
server_key_iv: "5c3746164d4153ab475386155714f547"
server_key_version: "encryptionkey@0"
I20260808 07:09:06.321079 5138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:06.332911 5138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:06.334898 5138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:06.335065 5138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:06.339383 5138 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33635
I20260808 07:09:06.339457 5239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33635 every 8 connection(s)
I20260808 07:09:06.339856 5138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:06.341697 5240 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:09:06.342401 5240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap starting.
I20260808 07:09:06.348703 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5138
I20260808 07:09:06.349085 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:46193
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=40185
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:06.352000 5240 log.cc:824] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:06.354287 5240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:06.354601 5240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap complete.
I20260808 07:09:06.356974 5240 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:06.357226 5240 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Initialized, Role: FOLLOWER
I20260808 07:09:06.357460 5240 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:06.357563 5240 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:06.357623 5240 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:06.357681 5240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:09:06.358680 5240 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:06.358803 5240 leader_election.cc:302] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59ad05803f84285a7ed988a92c309f4; no voters:
I20260808 07:09:06.359009 5240 leader_election.cc:288] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:09:06.359045 5244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:09:06.359170 5244 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 LEADER]: Becoming Leader. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Running, Role: LEADER
I20260808 07:09:06.359294 5240 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:06.359325 5244 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:06.359675 5245 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:06.359753 5245 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:06.360050 5246 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59ad05803f84285a7ed988a92c309f4. Latest consensus state: current_term: 2 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:06.360163 5246 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:06.360419 5250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:06.361287 5250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:06.361418 5250 catalog_manager.cc:1259] Loaded cluster ID: b71c57024afe4d1493d6997c28cfbfa5
I20260808 07:09:06.361503 5250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:09:06.361845 5250 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 07:09:06.366914 5250 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 07:09:06.369992 5250 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:06.370191 5250 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Loaded TSK: 0
I20260808 07:09:06.370406 5250 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:09:06.440682 5242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:06.440894 5242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:06.440918 5242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:06.442167 5242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:06.442214 5242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:06.442240 5242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:09:06.443732 5242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:46193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=40185
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:06.443976 5242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:06.444228 5242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:06.450397 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.450420 5278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.450423 5276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:06.450802 5242 server_base.cc:1034] running on GCE node
I20260808 07:09:06.451004 5242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:06.451251 5242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:06.452418 5242 hybrid_clock.cc:648] HybridClock initialized: now 1786172946452399 us; error 33 us; skew 500 ppm
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:06.455021 5242 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:09:06.455538 5242 webserver.cc:460] Webserver started at http://127.18.63.65:40185/ using document root <none> and password file <none>
I20260808 07:09:06.455791 5242 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:06.455852 5242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:06.457394 5242 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:06.458184 5286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.458350 5242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:09:06.458458 5242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "829e35f1d347ecc5e14ceb3a66b457cf"
server_key_iv: "1488830c773d5a74aba5c951566c7298"
server_key_version: "encryptionkey@0"
I20260808 07:09:06.458823 5242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:06.471232 5242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:06.473696 5242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:06.473865 5242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:06.474151 5242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:06.474701 5297 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:09:06.476189 5242 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:09:06.476243 5242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:09:06.476276 5242 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:09:06.476986 5242 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:09:06.477034 5242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:09:06.477082 5297 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap starting.
I20260808 07:09:06.485430 5242 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46193
I20260808 07:09:06.485643 5431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46193 every 8 connection(s)
I20260808 07:09:06.485948 5242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:09:06.490341 5297 log.cc:824] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Log is configured to *not* fsync() on all Append() calls
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:06.494213 5297 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:06.494519 5297 tablet_bootstrap.cc:492] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap complete.
I20260808 07:09:06.495839 5297 ts_tablet_manager.cc:1397] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.011s
I20260808 07:09:06.495909 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5242
I20260808 07:09:06.496235 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:37229
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=41179
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:06.497205 5297 raft_consensus.cc:348] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:06.497577 5297 raft_consensus.cc:729] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e07e1fcdf84c278fe56050c5e2d53c, State: Initialized, Role: FOLLOWER
I20260808 07:09:06.497733 5297 consensus_queue.cc:260] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } }
I20260808 07:09:06.498030 5297 ts_tablet_manager.cc:1428] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260808 07:09:06.498915 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.486782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:59209 (local address 127.18.63.126:33635)
0808 07:09:06.486993 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.487003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.487046 (+ 43us) server_negotiation.cc:407] Connection header received
0808 07:09:06.487798 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.487807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.487874 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.488053 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:06.488835 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.490315 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.491725 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.492098 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.492255 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.494646 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:06.494650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:06.494651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:06.494663 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:06.496886 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.497425 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.497429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.497429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.497554 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.497901 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.497901 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.497902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.498223 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:06.498303 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.498661 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.498773 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260808 07:09:06.499020 5435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.486663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:59209)
0808 07:09:06.486928 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.486966 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.487533 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.488089 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.488093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.488441 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:06.488695 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.488700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.490516 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.490517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.491624 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.491639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.492250 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.492252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.492400 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.492974 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:06.492983 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:06.494530 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:06.496992 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.496998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.497000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.497319 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.497670 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.497671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.497671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.497818 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.498296 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:06.498302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:06.498477 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.498603 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.498790 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":91,"threads_started":1}
I20260808 07:09:06.499794 5432 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:06.499905 5432 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:06.500092 5432 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:06.500844 5182 ts_manager.cc:194] Registered new tserver with Master: 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193)
I20260808 07:09:06.501914 5182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:59209
I20260808 07:09:06.502887 5432 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:09:06.510349 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.502186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.18.63.65:46193)
0808 07:09:06.502371 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.502376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.502388 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:09:06.502426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.502429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.502447 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.502526 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:06.503059 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.503875 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.504698 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.504988 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.505071 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.507333 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:06.507341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:06.507341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:06.507348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:06.509027 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.509446 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.509449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.509450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.509513 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.509741 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.509742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.509742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.509876 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:06.509971 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.510162 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.510248 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260808 07:09:06.510349 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.501476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46193 (local address 127.0.0.1:34486)
0808 07:09:06.501666 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.501709 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.501779 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.502547 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.502553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.502922 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:06.502981 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.502984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.503963 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.503964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.504582 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.504584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.505070 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.505071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.505150 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.505682 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:06.505688 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:06.507239 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:06.509124 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.509130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.509132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.509374 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.509592 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.509593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.509593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.509669 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.509952 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:06.509959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:06.510012 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.510115 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.510248 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260808 07:09:06.510954 5348 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b692970b2141f99344a0cf6e2de589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:06 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:34486
I20260808 07:09:06.511478 5442 tablet_replica.cc:330] stopping tablet replica
I20260808 07:09:06.511607 5442 raft_consensus.cc:2229] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:06.511703 5442 raft_consensus.cc:2258] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:06.512018 5442 ts_tablet_manager.cc:1905] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:06.513947 5442 ts_tablet_manager.cc:1918] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:06.514011 5442 log.cc:1196] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/73b692970b2141f99344a0cf6e2de589
I20260808 07:09:06.514190 5442 ts_tablet_manager.cc:1939] T 73b692970b2141f99344a0cf6e2de589 P 58e07e1fcdf84c278fe56050c5e2d53c: Deleting consensus metadata
I20260808 07:09:06.514705 5160 catalog_manager.cc:4841] TS 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193): tablet 73b692970b2141f99344a0cf6e2de589 (table test-table [id=9210a2a46f7b4315877be601a1b9d421]) successfully deleted
W20260808 07:09:06.584772 5439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:06.584991 5439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:06.585016 5439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:06.586208 5439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:06.586251 5439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:06.586277 5439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:09:06.587697 5439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:37229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=41179
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:06.587881 5439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:06.588065 5439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:06.593297 5451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.593258 5450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.593274 5453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:06.593751 5439 server_base.cc:1034] running on GCE node
I20260808 07:09:06.593961 5439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:06.594219 5439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:06.595379 5439 hybrid_clock.cc:648] HybridClock initialized: now 1786172946595360 us; error 36 us; skew 500 ppm
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:06.598083 5439 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:09:06.598623 5439 webserver.cc:460] Webserver started at http://127.18.63.66:41179/ using document root <none> and password file <none>
I20260808 07:09:06.598840 5439 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:06.598893 5439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:06.600183 5439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:09:06.600759 5462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.600999 5439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:09:06.601114 5439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "208a7a6fcdfa8df36e1f2d1bad72413a"
server_key_iv: "c9c9d39e0c0b672f495bedcf23fca760"
server_key_version: "encryptionkey@0"
I20260808 07:09:06.601576 5439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:06.616721 5439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:06.619194 5439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:06.619361 5439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:06.619618 5439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:06.619932 5439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:06.619992 5439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.620033 5439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:06.620061 5439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.626647 5439 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:37229
I20260808 07:09:06.626708 5615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:37229 every 8 connection(s)
I20260808 07:09:06.627053 5439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:06.633862 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5439
I20260808 07:09:06.634143 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:46195
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=42567
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:06.636926 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.627729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43279 (local address 127.18.63.126:33635)
0808 07:09:06.627800 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.627802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.628087 (+ 285us) server_negotiation.cc:407] Connection header received
0808 07:09:06.628645 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.628646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.628674 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.628739 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:06.629702 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.630222 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.631242 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.631454 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.631542 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.633733 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:06.633734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:06.633735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:06.633742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:06.635344 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.635864 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.635865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.635865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.635942 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.636233 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.636234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.636234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.636392 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:06.636466 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.636709 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.636816 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:09:06.636948 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.627740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:43279)
0808 07:09:06.627929 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.627964 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.628527 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.628817 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.628820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.629401 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:06.629595 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.629600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.630324 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.630325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.631142 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.631145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.631555 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.631556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.631684 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.632150 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:06.632158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:06.633621 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:06.635430 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.635439 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.635441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.635770 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.636023 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.636024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.636024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.636165 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.636461 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:06.636466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:06.636578 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.636664 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.636784 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 07:09:06.637568 5616 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:06.637673 5616 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:06.637877 5616 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:06.638283 5182 ts_manager.cc:194] Registered new tserver with Master: b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
I20260808 07:09:06.638669 5182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:43279
W20260808 07:09:06.719115 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:06.719312 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:06.719336 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:06.720470 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:06.720512 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:06.720562 5621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:09:06.722102 5621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:46195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=42567
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:06.722337 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:06.722571 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:06.727952 5628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.727950 5631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:06.727946 5629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:06.728368 5621 server_base.cc:1034] running on GCE node
I20260808 07:09:06.728575 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:06.728775 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:06.729934 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786172946729929 us; error 39 us; skew 500 ppm
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:06.732409 5621 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:09:06.732856 5621 webserver.cc:460] Webserver started at http://127.18.63.67:42567/ using document root <none> and password file <none>
I20260808 07:09:06.733096 5621 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:06.733175 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:06.734557 5621 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:06.735172 5640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.735356 5621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:06.735458 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "13684f39a48943529863a9a727f034d5"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "5778ea792251d32f42cc4e5065770d1b"
server_key_iv: "f34957751289e1ffc528e43016d0052f"
server_key_version: "encryptionkey@0"
I20260808 07:09:06.735828 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:06.762666 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:06.764672 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:06.764844 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:06.765101 5621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:06.765489 5621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:06.765552 5621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.765589 5621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:06.765616 5621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:06.771718 5621 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46195
I20260808 07:09:06.771777 5793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46195 every 8 connection(s)
I20260808 07:09:06.772120 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:09:06 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172946, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:06.781311 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5621
I20260808 07:09:06.781419 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.772842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52447 (local address 127.18.63.126:33635)
0808 07:09:06.772935 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.772938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.773138 (+ 200us) server_negotiation.cc:407] Connection header received
0808 07:09:06.773686 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.773687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.773698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.773748 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:06.774677 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.775160 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.776134 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.776318 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.776369 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.778672 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:06.778674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:06.778674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:06.778681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:06.780150 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.780500 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.780501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.780501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.780554 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.780765 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.780766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.780766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.780871 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:06.780922 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.781189 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.781279 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:09:06.781574 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.772780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:52447)
0808 07:09:06.772969 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.773008 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.773579 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.773809 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.773812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.774394 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:06.774586 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.774591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.775241 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.775242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.776053 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.776055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.776393 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.776394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.776493 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.777053 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:06.777061 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:06.778573 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:06.780220 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.780226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.780228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.780435 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.780612 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:06.780613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:06.780613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:06.780710 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:06.780926 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:06.780930 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:06.781044 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.781135 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.781271 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260808 07:09:06.782166 5795 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33635
I20260808 07:09:06.782280 5795 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:06.782492 5795 heartbeater.cc:511] Master 127.18.63.126:33635 requested a full tablet report, sending...
I20260808 07:09:06.782939 5182 ts_manager.cc:194] Registered new tserver with Master: 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195)
I20260808 07:09:06.783414 5182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:52447
I20260808 07:09:06.788340 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.781823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.18.63.126:33635)
0808 07:09:06.781936 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.781938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.781947 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:06.782079 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.782080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.782092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.782164 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:06.782927 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.783632 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.784389 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.784603 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.784681 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.785950 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:06.785952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:06.785952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:06.785959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:06.787181 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.787539 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.787540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.787541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.787598 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:06.787810 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:06.787811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:06.787811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:06.787923 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:06.788103 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.788152 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.788231 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260808 07:09:06.789093 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:09:06.791709 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.789433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.18.63.126:33635)
0808 07:09:06.789470 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.789472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.789668 (+ 196us) server_negotiation.cc:407] Connection header received
0808 07:09:06.789720 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.789721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.789748 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.789805 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:06.789896 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.790327 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.790997 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.791181 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.791232 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.791346 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:06.791429 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:09:06.791481 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.791511 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.791526 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:09:06.791999 5182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:09:06.793783 5182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:09:06.796022 5348 tablet_service.cc:1467] Processing CreateTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 (DEFAULT_TABLE table=test-table [id=d2a1afa71aef4100ab52ae82a41c9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:06.796253 5348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d0979fe1964b9fba8fa5b784dd7166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:06.797906 5297 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap starting.
I20260808 07:09:06.798242 5297 tablet_bootstrap.cc:654] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:06.799448 5297 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: No bootstrap required, opened a new log
I20260808 07:09:06.799513 5297 ts_tablet_manager.cc:1397] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:09:06.799702 5297 raft_consensus.cc:348] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.799795 5297 raft_consensus.cc:374] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:06.799847 5297 raft_consensus.cc:729] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e07e1fcdf84c278fe56050c5e2d53c, State: Initialized, Role: FOLLOWER
I20260808 07:09:06.799855 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.795619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46195 (local address 127.0.0.1:49130)
0808 07:09:06.795808 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.795817 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.795853 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.796179 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.796180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.796182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:06.796224 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.796226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.797248 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.797249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.798491 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.798494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.799597 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.799598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.799675 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.799699 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.799735 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.799774 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":17,"thread_start_us":88,"threads_started":1}
I20260808 07:09:06.799899 5297 consensus_queue.cc:260] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.799975 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.795612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:37229 (local address 127.0.0.1:49472)
0808 07:09:06.795670 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:06.795680 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:06.795720 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:06.796180 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:06.796182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:06.796184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:06.796229 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.796231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.797577 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.797579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.798613 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:06.798616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.799689 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.799690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:06.799757 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.799761 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:06.799809 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:09:06.799862 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 07:09:06.800014 5297 ts_tablet_manager.cc:1428] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:06.800287 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.795817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.18.63.67:46195)
0808 07:09:06.795978 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.795982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.795991 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:06.796033 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.796035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.796046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.796161 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:06.796289 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.797145 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.798592 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.799487 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.800031 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.800099 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.800126 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.800170 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260808 07:09:06.800539 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:06.795753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.18.63.66:37229)
0808 07:09:06.795964 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:06.795969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:06.795979 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:09:06.796021 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:06.796023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:06.796040 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:06.796316 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:06.796398 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.797471 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.798699 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:06.799597 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:06.799676 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:06.799737 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:06.800407 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 07:09:06.800462 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260808 07:09:06.801026 5702 tablet_service.cc:1467] Processing CreateTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 (DEFAULT_TABLE table=test-table [id=d2a1afa71aef4100ab52ae82a41c9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:06.801240 5525 tablet_service.cc:1467] Processing CreateTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 (DEFAULT_TABLE table=test-table [id=d2a1afa71aef4100ab52ae82a41c9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:06.801429 5702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d0979fe1964b9fba8fa5b784dd7166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:06.801620 5525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d0979fe1964b9fba8fa5b784dd7166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:06.804491 5809 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Bootstrap starting.
I20260808 07:09:06.804507 5813 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Bootstrap starting.
I20260808 07:09:06.805130 5813 tablet_bootstrap.cc:654] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:06.805272 5809 tablet_bootstrap.cc:654] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:06.805807 5813 log.cc:824] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:06.805919 5809 log.cc:824] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:06.806788 5813 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: No bootstrap required, opened a new log
I20260808 07:09:06.806870 5813 ts_tablet_manager.cc:1397] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:09:06.807044 5809 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: No bootstrap required, opened a new log
I20260808 07:09:06.807125 5809 ts_tablet_manager.cc:1397] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:09:06.808249 5813 raft_consensus.cc:348] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.808382 5813 raft_consensus.cc:374] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:06.808403 5813 raft_consensus.cc:729] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13684f39a48943529863a9a727f034d5, State: Initialized, Role: FOLLOWER
I20260808 07:09:06.808518 5813 consensus_queue.cc:260] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.808566 5809 raft_consensus.cc:348] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.808755 5809 raft_consensus.cc:374] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:06.808801 5809 raft_consensus.cc:729] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41a27da1ced4afc841f9c3c86ea88e1, State: Initialized, Role: FOLLOWER
I20260808 07:09:06.808810 5795 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
I20260808 07:09:06.808912 5809 consensus_queue.cc:260] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:06.809072 5813 ts_tablet_manager.cc:1428] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:09:06.809196 5809 ts_tablet_manager.cc:1428] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:09:06.809319 5616 heartbeater.cc:503] Master 127.18.63.126:33635 was elected leader, sending a full tablet report...
W20260808 07:09:06.877679 5617 tablet.cc:2369] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:09:06.986843 5433 tablet.cc:2369] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:09:07.019259 5816 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:07.019404 5816 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:07.019912 5816 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229), 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193)
W20260808 07:09:07.022682 5796 tablet.cc:2369] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:09:07.022977 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.020128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46193 (local address 127.18.63.67:32831)
0808 07:09:07.020207 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.020220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:07.020271 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:07.020520 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:07.020521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:07.020523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:07.020605 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:07.020607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.021382 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.021383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:07.022133 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:07.022135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.022731 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:07.022814 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:07.022818 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:07.022859 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 07:09:07.022898 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 07:09:07.023146 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.020229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:53217 (local address 127.18.63.66:37229)
0808 07:09:07.020326 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:07.020328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:07.020422 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:09:07.020523 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:07.020525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:07.020537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:07.020591 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:07.020799 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.021271 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022278 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.022736 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022807 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:07.022816 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:07.023028 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 07:09:07.023082 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 07:09:07.023192 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.020155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:37229 (local address 127.18.63.67:53217)
0808 07:09:07.020326 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.020339 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:07.020426 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:07.020620 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:07.020621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:07.020622 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:07.020713 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:07.020714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.021413 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.021414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:07.022166 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:07.022191 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.022852 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:07.022936 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:07.022940 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:07.023022 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:09:07.023063 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260808 07:09:07.023305 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.020219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:32831 (local address 127.18.63.65:46193)
0808 07:09:07.020341 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:07.020343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:07.020353 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:09:07.020389 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:07.020390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:07.020420 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:07.020503 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:07.020701 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.021261 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022222 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.022643 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.022735 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:07.022923 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:07.022944 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 07:09:07.022983 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260808 07:09:07.023650 5375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d0979fe1964b9fba8fa5b784dd7166" candidate_uuid: "13684f39a48943529863a9a727f034d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" is_pre_election: true
I20260808 07:09:07.023679 5557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d0979fe1964b9fba8fa5b784dd7166" candidate_uuid: "13684f39a48943529863a9a727f034d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" is_pre_election: true
I20260808 07:09:07.023819 5375 raft_consensus.cc:2454] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13684f39a48943529863a9a727f034d5 in term 0.
I20260808 07:09:07.023819 5557 raft_consensus.cc:2454] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13684f39a48943529863a9a727f034d5 in term 0.
I20260808 07:09:07.024031 5642 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1; no voters:
I20260808 07:09:07.024212 5816 raft_consensus.cc:2781] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:09:07.024284 5816 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:09:07.024312 5816 raft_consensus.cc:3037] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:07.025094 5816 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:07.025246 5816 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [CANDIDATE]: Term 1 election: Requested vote from peers b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229), 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193)
I20260808 07:09:07.025399 5557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d0979fe1964b9fba8fa5b784dd7166" candidate_uuid: "13684f39a48943529863a9a727f034d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
I20260808 07:09:07.025421 5375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d0979fe1964b9fba8fa5b784dd7166" candidate_uuid: "13684f39a48943529863a9a727f034d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
I20260808 07:09:07.025488 5375 raft_consensus.cc:3037] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:07.025475 5557 raft_consensus.cc:3037] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:07.026377 5375 raft_consensus.cc:2454] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13684f39a48943529863a9a727f034d5 in term 1.
I20260808 07:09:07.026377 5557 raft_consensus.cc:2454] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13684f39a48943529863a9a727f034d5 in term 1.
I20260808 07:09:07.026584 5641 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13684f39a48943529863a9a727f034d5, 58e07e1fcdf84c278fe56050c5e2d53c; no voters:
I20260808 07:09:07.026712 5816 raft_consensus.cc:2781] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:07.026868 5816 raft_consensus.cc:686] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 1 LEADER]: Becoming Leader. State: Replica: 13684f39a48943529863a9a727f034d5, State: Running, Role: LEADER
I20260808 07:09:07.026974 5816 consensus_queue.cc:237] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:07.027894 5182 catalog_manager.cc:5495] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13684f39a48943529863a9a727f034d5 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "13684f39a48943529863a9a727f034d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } health_report { overall_health: UNKNOWN } } }
I20260808 07:09:07.084846 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.082485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.18.63.67:46195)
0808 07:09:07.082553 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:07.082559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:07.082569 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:09:07.082630 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:07.082631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:07.082650 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:07.082708 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:07.082839 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.083330 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.084090 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:07.084292 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:07.084345 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:07.084486 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:07.084588 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:09:07.084649 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:07.084686 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:09:07.084705 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:09:07.087486 5375 raft_consensus.cc:1261] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Refusing update from remote peer 13684f39a48943529863a9a727f034d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:07.087879 5821 consensus_queue.cc:1035] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:07.089880 5557 raft_consensus.cc:1261] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Refusing update from remote peer 13684f39a48943529863a9a727f034d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:07.090304 5816 consensus_queue.cc:1035] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:07.092146 5825 mvcc.cc:205] Tried to move back new op lower bound from 7316164391266897920 to 7316164391023304704. Current Snapshot: MvccSnapshot[applied={T|T < 7316164391266897920}]
I20260808 07:09:07.094355 5181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:07.094440 5181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:07.095328 5181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Sending DeleteTablet for 3 replicas of tablet f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:07.095685 5702 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:07 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49130
I20260808 07:09:07.095767 5348 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:07 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:34486
I20260808 07:09:07.095788 5525 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d0979fe1964b9fba8fa5b784dd7166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:07 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49472
I20260808 07:09:07.095932 5837 tablet_replica.cc:330] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: stopping tablet replica
I20260808 07:09:07.096055 5837 raft_consensus.cc:2229] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:09:07.096135 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 5242
I20260808 07:09:07.096200 5837 raft_consensus.cc:2258] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:07.103905 5842 tablet_replica.cc:330] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: stopping tablet replica
I20260808 07:09:07.104048 5837 ts_tablet_manager.cc:1905] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:07.104264 5842 raft_consensus.cc:2229] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:07.104516 5842 raft_consensus.cc:2258] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:07.105072 5842 ts_tablet_manager.cc:1905] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:07.106418 5837 ts_tablet_manager.cc:1918] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:07.106519 5837 log.cc:1196] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:07.106833 5837 ts_tablet_manager.cc:1939] T f2d0979fe1964b9fba8fa5b784dd7166 P 13684f39a48943529863a9a727f034d5: Deleting consensus metadata
I20260808 07:09:07.107151 5842 ts_tablet_manager.cc:1918] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:07.107295 5842 log.cc:1196] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:07.107293 5163 catalog_manager.cc:4841] TS 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): tablet f2d0979fe1964b9fba8fa5b784dd7166 (table test-table [id=d2a1afa71aef4100ab52ae82a41c9d9d]) successfully deleted
I20260808 07:09:07.107664 5842 ts_tablet_manager.cc:1939] T f2d0979fe1964b9fba8fa5b784dd7166 P b41a27da1ced4afc841f9c3c86ea88e1: Deleting consensus metadata
I20260808 07:09:07.108167 5161 catalog_manager.cc:4841] TS b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): tablet f2d0979fe1964b9fba8fa5b784dd7166 (table test-table [id=d2a1afa71aef4100ab52ae82a41c9d9d]) successfully deleted
W20260808 07:09:07.110822 5160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:46193 (error 108)
I20260808 07:09:07.111168 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 5439
I20260808 07:09:07.111364 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.111242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46193 (local address 127.0.0.1:34500)
0808 07:09:07.111297 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.111330 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:09:07.113101 5160 catalog_manager.cc:4584] TS 58e07e1fcdf84c278fe56050c5e2d53c (127.18.63.65:46193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2d0979fe1964b9fba8fa5b784dd7166: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46193: connect: Connection refused (error 111)
I20260808 07:09:07.119725 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 5621
I20260808 07:09:07.128284 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 5138
I20260808 07:09:07.139081 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--webserver_interface=127.18.63.126
--webserver_port=37269
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:07.224792 5845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:07.225009 5845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:07.225032 5845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:07.226238 5845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:07.226284 5845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:07.226298 5845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:07.226311 5845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:07.226375 5845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:07.227849 5845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=37269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:07.228089 5845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:07.228323 5845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:07.233475 5854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.233592 5845 server_base.cc:1034] running on GCE node
W20260808 07:09:07.233470 5857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.233492 5852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.233945 5845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:07.234264 5845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:07.235443 5845 hybrid_clock.cc:648] HybridClock initialized: now 1786172947235425 us; error 36 us; skew 500 ppm
Aug 08 07:09:07 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:07.237948 5845 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:07.238389 5845 webserver.cc:460] Webserver started at http://127.18.63.126:37269/ using document root <none> and password file <none>
I20260808 07:09:07.238610 5845 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:07.238686 5845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:07.240046 5845 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.240690 5866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.240863 5845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:07.240968 5845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "d59ad05803f84285a7ed988a92c309f4"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "e254b0b38f1867e617d246c46104c0c5"
server_key_iv: "5c3746164d4153ab475386155714f547"
server_key_version: "encryptionkey@0"
I20260808 07:09:07.241371 5845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:07.250818 5845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:07.252581 5845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:07.252748 5845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:07.256892 5845 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33635
I20260808 07:09:07.256939 5937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33635 every 8 connection(s)
I20260808 07:09:07.257378 5845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:07.259018 5938 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:09:07.259636 5938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap starting.
I20260808 07:09:07.264810 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5845
I20260808 07:09:07.265202 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:46193
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=40185
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:07.269646 5938 log.cc:824] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:07.272583 5938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:07.272966 5938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: Bootstrap complete.
I20260808 07:09:07.275635 5938 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:07.275892 5938 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Initialized, Role: FOLLOWER
I20260808 07:09:07.276084 5938 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:07.276160 5938 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:07.276234 5938 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:07.276295 5938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:09:07.277493 5938 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:07.277616 5938 leader_election.cc:302] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59ad05803f84285a7ed988a92c309f4; no voters:
I20260808 07:09:07.277841 5938 leader_election.cc:288] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:09:07.277879 5942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:09:07.278072 5942 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [term 3 LEADER]: Becoming Leader. State: Replica: d59ad05803f84285a7ed988a92c309f4, State: Running, Role: LEADER
I20260808 07:09:07.278160 5938 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:07.278898 5942 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } }
I20260808 07:09:07.279634 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:07.279726 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:07.280153 5945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59ad05803f84285a7ed988a92c309f4. Latest consensus state: current_term: 3 leader_uuid: "d59ad05803f84285a7ed988a92c309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59ad05803f84285a7ed988a92c309f4" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33635 } } }
I20260808 07:09:07.280328 5945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:07.280519 5963 catalog_manager.cc:1259] Loaded cluster ID: b71c57024afe4d1493d6997c28cfbfa5
I20260808 07:09:07.280582 5963 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: loading cluster ID for follower catalog manager: success
W20260808 07:09:07.280926 5963 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d59ad05803f84285a7ed988a92c309f4: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260808 07:09:07.281006 5963 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260808 07:09:07.281097 5966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:07.281565 5966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:07.281680 5966 catalog_manager.cc:1259] Loaded cluster ID: b71c57024afe4d1493d6997c28cfbfa5
I20260808 07:09:07.281718 5966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 07:09:07.281916 5966 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8b27a0d9ad google::LogMessage::Fail() at ??:0
@ 0x7f8b27a1088a google::LogMessage::SendToLog() at ??:0
@ 0x7f8b27a0d4de google::LogMessage::Flush() at ??:0
@ 0x7f8b27a0ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8b2f92e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f8b2f9378b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8b298db0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8b298d0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8b2a7346db start_thread at ??:0
@ 0x7f8b2650b61f clone at ??:0
W20260808 07:09:07.372193 5940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:07.372468 5940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:07.372516 5940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:07.373895 5940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:07.373978 5940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:07.374030 5940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:09:07.375736 5940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:46193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=40185
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:07.376078 5940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:07.376358 5940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:07.382494 5990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.382457 5987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.382465 5986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.383024 5940 server_base.cc:1034] running on GCE node
I20260808 07:09:07.383409 5940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:07.383746 5940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:07.384945 5940 hybrid_clock.cc:648] HybridClock initialized: now 1786172947384929 us; error 32 us; skew 500 ppm
Aug 08 07:09:07 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:07.388271 5940 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:09:07.388752 5940 webserver.cc:460] Webserver started at http://127.18.63.65:40185/ using document root <none> and password file <none>
I20260808 07:09:07.388993 5940 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:07.389042 5940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:07.390619 5940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.391427 5998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.391839 5940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.391955 5940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "58e07e1fcdf84c278fe56050c5e2d53c"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "829e35f1d347ecc5e14ceb3a66b457cf"
server_key_iv: "1488830c773d5a74aba5c951566c7298"
server_key_version: "encryptionkey@0"
I20260808 07:09:07.392334 5940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:07.418536 5940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:07.422261 5940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:07.422566 5940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:07.422940 5940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:07.423656 6006 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:09:07.425375 5940 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:09:07.425457 5940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:09:07.425522 5940 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:09:07.426697 5940 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:09:07.426759 5940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:09:07.426826 6006 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap starting.
I20260808 07:09:07.436554 5940 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46193
I20260808 07:09:07.436761 6142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46193 every 8 connection(s)
I20260808 07:09:07.437296 5940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:09:07.439045 6006 log.cc:824] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:07.441473 6006 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:07.441764 6006 tablet_bootstrap.cc:492] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Bootstrap complete.
I20260808 07:09:07.442734 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 5940
I20260808 07:09:07.442816 6006 ts_tablet_manager.cc:1397] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260808 07:09:07.443202 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:37229
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=41179
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:07.443974 6006 raft_consensus.cc:348] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:07.444239 6006 raft_consensus.cc:729] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e07e1fcdf84c278fe56050c5e2d53c, State: Initialized, Role: FOLLOWER
I20260808 07:09:07.444409 6006 consensus_queue.cc:260] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:07.444691 6006 ts_tablet_manager.cc:1428] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260808 07:09:07.566152 6149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:07.566416 6149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:07.566463 6149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:07.567701 6149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:07.567778 6149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:07.567827 6149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:09:07.569494 6149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:37229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=41179
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:07.569785 6149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:07.570092 6149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:07.570297 6146 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:09:07.438367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:58827)
0808 07:09:07.438670 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.438749 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:07.439532 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:07.570087 (+130555us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":123,"threads_started":1}
W20260808 07:09:07.570662 6143 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33635 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:33635: Client connection negotiation failed: client connection to 127.18.63.126:33635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:09:07.571082 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.570918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:44941)
0808 07:09:07.570978 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.571055 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:09:07.571503 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.571389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:38597)
0808 07:09:07.571441 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.571480 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:09:07.571642 6143 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:09:07.575743 6161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.575743 6155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.575737 6157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.575899 6149 server_base.cc:1034] running on GCE node
I20260808 07:09:07.576087 6149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:07.576344 6149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:07.577505 6149 hybrid_clock.cc:648] HybridClock initialized: now 1786172947577499 us; error 36 us; skew 500 ppm
Aug 08 07:09:07 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:07.580098 6149 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:09:07.580567 6149 webserver.cc:460] Webserver started at http://127.18.63.66:41179/ using document root <none> and password file <none>
I20260808 07:09:07.580823 6149 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:07.580897 6149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:07.582360 6149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.582957 6173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.583141 6149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:07.583249 6149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "b41a27da1ced4afc841f9c3c86ea88e1"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "208a7a6fcdfa8df36e1f2d1bad72413a"
server_key_iv: "c9c9d39e0c0b672f495bedcf23fca760"
server_key_version: "encryptionkey@0"
I20260808 07:09:07.583613 6149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:07.594606 6149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:07.597318 6149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:07.597537 6149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:07.597779 6149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:07.598094 6149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:07.598152 6149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.598201 6149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:07.598224 6149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.604456 6149 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:37229
I20260808 07:09:07.604513 6317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:37229 every 8 connection(s)
I20260808 07:09:07.604903 6149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:09:07.606005 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.605644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:37077)
0808 07:09:07.605796 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.605869 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
W20260808 07:09:07.606426 6319 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33635 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:33635: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
I20260808 07:09:07.606709 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.606620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:50041)
0808 07:09:07.606665 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.606692 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:09:07.606976 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.606902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:45809)
0808 07:09:07.606941 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.606961 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 07:09:07.607088 6319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:09:07.611217 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 6149
I20260808 07:09:07.611470 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:46195
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:33635
--webserver_port=42567
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:07.696095 6322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:07.696314 6322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:07.696338 6322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:07.697585 6322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:07.697633 6322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:07.697660 6322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:09:07.699174 6322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:46195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=42567
--tserver_master_addrs=127.18.63.126:33635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:07.699391 6322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:07.699574 6322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:07.704913 6332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:07.704902 6329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.705093 6322 server_base.cc:1034] running on GCE node
W20260808 07:09:07.704897 6330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:07.705430 6322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:07.705703 6322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:07.706858 6322 hybrid_clock.cc:648] HybridClock initialized: now 1786172947706847 us; error 47 us; skew 500 ppm
Aug 08 07:09:07 dist-test-slave-bw88 krb5kdc[4381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:07.709420 6322 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:09:07.709985 6322 webserver.cc:460] Webserver started at http://127.18.63.67:42567/ using document root <none> and password file <none>
I20260808 07:09:07.710227 6322 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:07.710296 6322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:07.711664 6322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.712344 6342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.712522 6322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:07.712627 6322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "13684f39a48943529863a9a727f034d5"
format_stamp: "Formatted at 2026-08-08 07:09:05 on dist-test-slave-bw88"
server_key: "5778ea792251d32f42cc4e5065770d1b"
server_key_iv: "f34957751289e1ffc528e43016d0052f"
server_key_version: "encryptionkey@0"
I20260808 07:09:07.712994 6322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:07.721521 6322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:07.723294 6322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:07.723459 6322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:07.723701 6322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:07.724010 6322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:07.724066 6322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.724105 6322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:07.724141 6322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:07.730530 6322 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46195
I20260808 07:09:07.730581 6480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46195 every 8 connection(s)
I20260808 07:09:07.730927 6322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:09:07.731971 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.731584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:42837)
0808 07:09:07.731749 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.731829 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260808 07:09:07.732386 6481 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:33635 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:33635: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
I20260808 07:09:07.732681 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.732590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:34121)
0808 07:09:07.732637 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.732665 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 07:09:07.732936 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:07.732887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:43551)
0808 07:09:07.732908 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:07.732925 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260808 07:09:07.733059 6481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:09:07.737309 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 6322
I20260808 07:09:08.572247 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.571985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:40677)
0808 07:09:08.572162 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:08.572197 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260808 07:09:08.607637 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.607409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:54377)
0808 07:09:08.607546 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:08.607577 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260808 07:09:08.733744 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.733524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:34643)
0808 07:09:08.733659 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:08.733695 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260808 07:09:08.743366 6495 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:08.743532 6495 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:08.743942 6495 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
Aug 08 07:09:08 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.67@KRBTEST.COM
Aug 08 07:09:08 dist-test-slave-bw88 krb5kdc[4381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.66@KRBTEST.COM
I20260808 07:09:08.754984 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.744118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46195 (local address 127.18.63.65:38241)
0808 07:09:08.744180 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:08.744192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:08.744254 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:08.745183 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:08.745186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:08.745732 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:08.745921 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:08.745927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.747012 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.747013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:08.747823 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:08.747825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.748433 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.748434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:08.748547 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:08.749281 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:08.749285 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:08.751049 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:08.753158 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:08.753165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:08.753167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:08.753480 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:08.753737 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:08.753738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:08.753739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:08.753885 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:08.754394 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:08.754400 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:08.754554 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 07:09:08.754683 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 07:09:08.754854 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":2}
I20260808 07:09:08.755031 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.744252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:38241 (local address 127.18.63.67:46195)
0808 07:09:08.744438 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:08.744449 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:08.744480 (+ 31us) server_negotiation.cc:407] Connection header received
0808 07:09:08.744929 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:08.744931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:08.744982 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:08.745105 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:08.746050 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.746903 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.747917 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.748334 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.748459 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:08.751171 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:08.751175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:08.751175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:08.751185 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:08.753074 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:08.753563 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:08.753565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:08.753566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:08.753661 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:08.753967 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:08.753968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:08.753968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:08.754298 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:08.754399 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:08.754757 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 07:09:08.754896 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":71,"threads_started":1}
I20260808 07:09:08.755332 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.744278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:52233 (local address 127.18.63.66:37229)
0808 07:09:08.744483 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:08.744494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:08.744530 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:09:08.744997 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:08.745000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:08.745038 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:08.745208 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:08.746050 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.746910 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.747941 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.748334 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.748460 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:08.751753 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:08.751756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:08.751757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:08.751766 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:08.753807 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:08.754205 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:08.754207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:08.754208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:08.754285 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:08.754496 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:08.754497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:08.754497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:08.754786 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:08.754869 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:08.755068 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 07:09:08.755237 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
I20260808 07:09:08.755178 6498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:08.744118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:37229 (local address 127.18.63.65:52233)
0808 07:09:08.744411 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:08.744423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:08.744476 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:08.745217 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:08.745218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:08.745733 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:08.745921 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:08.745927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.747037 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.747038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:08.747824 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:08.747827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:08.748437 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:08.748438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:08.748547 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:08.749224 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:08.749233 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:08.751446 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:08.753886 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:08.753887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:08.753888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:08.754120 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:08.754357 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:08.754358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:08.754359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:08.754429 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:08.754868 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:08.754869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:08.754912 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:09:08.754988 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:09:08.755079 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":246,"thread_start_us":67,"threads_started":1}
W20260808 07:09:08.756446 6000 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
W20260808 07:09:08.756529 6001 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:08.756584 6001 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c; no voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1
I20260808 07:09:08.756744 6495 raft_consensus.cc:2726] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:09:09.572980 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:09.572731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:51023)
0808 07:09:09.572901 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:09.572931 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260808 07:09:09.608328 6500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:09.608096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:54617)
0808 07:09:09.608247 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:09.608287 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260808 07:09:09.734431 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:09.734198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:43677)
0808 07:09:09.734357 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:09.734393 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260808 07:09:10.315379 6502 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:10.315512 6502 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:10.315733 6502 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
W20260808 07:09:10.316136 6000 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
W20260808 07:09:10.316222 6001 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:10.316306 6000 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c; no voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1
I20260808 07:09:10.316395 6502 raft_consensus.cc:2726] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:09:10.573930 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:10.573663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:34021)
0808 07:09:10.573815 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:10.573859 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260808 07:09:10.609038 6504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:10.608817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:56381)
0808 07:09:10.608953 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:10.608993 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260808 07:09:10.735175 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:10.734932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:44977)
0808 07:09:10.735094 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:10.735134 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260808 07:09:11.574580 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:11.574371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:32969)
0808 07:09:11.574498 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:11.574533 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 07:09:11.609822 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:11.609574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:58439)
0808 07:09:11.609714 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:11.609767 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260808 07:09:11.735968 6508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:11.735745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:58945)
0808 07:09:11.735887 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:11.735925 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 07:09:12.575387 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:12.575150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:51477)
0808 07:09:12.575299 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:12.575338 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260808 07:09:12.610669 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:12.610336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:46669)
0808 07:09:12.610553 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:12.610596 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":171,"threads_started":1}
I20260808 07:09:12.736604 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:12.736389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:57449)
0808 07:09:12.736532 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:12.736566 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260808 07:09:13.351990 6514 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:13.352084 6514 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:13.352308 6514 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
W20260808 07:09:13.352718 6001 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
W20260808 07:09:13.352828 6000 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:13.352912 6000 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c; no voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1
I20260808 07:09:13.353000 6514 raft_consensus.cc:2726] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:09:13.576206 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:13.575935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:50781)
0808 07:09:13.576101 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:13.576158 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260808 07:09:13.611531 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:13.611242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:47551)
0808 07:09:13.611389 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:13.611451 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260808 07:09:13.737273 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:13.737058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:42751)
0808 07:09:13.737197 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:13.737230 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 07:09:14.576967 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:14.576731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:48535)
0808 07:09:14.576875 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:14.576918 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260808 07:09:14.612205 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:14.611989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:43297)
0808 07:09:14.612119 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:14.612159 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260808 07:09:14.738025 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:14.737795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:36577)
0808 07:09:14.737929 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:14.737964 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260808 07:09:15.321938 6524 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:15.322045 6524 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:15.322293 6524 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
W20260808 07:09:15.322741 6000 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
W20260808 07:09:15.322825 6001 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:15.322906 6001 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c; no voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1
I20260808 07:09:15.323002 6524 raft_consensus.cc:2726] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:09:15.577942 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:15.577730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:51519)
0808 07:09:15.577860 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:15.577895 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260808 07:09:15.612879 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:15.612631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:57535)
0808 07:09:15.612771 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:15.612806 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260808 07:09:15.738751 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:15.738526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:38967)
0808 07:09:15.738653 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:15.738694 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260808 07:09:16.578691 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:16.578427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:51333)
0808 07:09:16.578599 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:16.578641 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260808 07:09:16.613694 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:16.613426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:40803)
0808 07:09:16.613589 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:16.613645 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
I20260808 07:09:16.739439 6532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:16.739217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:40367)
0808 07:09:16.739351 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:16.739392 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260808 07:09:17.579460 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:17.579239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:53909)
0808 07:09:17.579380 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:17.579413 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260808 07:09:17.614353 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:17.614132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:44993)
0808 07:09:17.614272 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:17.614304 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260808 07:09:17.740232 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:17.739969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:60865)
0808 07:09:17.740119 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:17.740158 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260808 07:09:18.580139 6536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:18.579888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:49553)
0808 07:09:18.580042 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:18.580087 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260808 07:09:18.615048 6537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:18.614806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:45059)
0808 07:09:18.614958 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:18.615000 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260808 07:09:18.664710 6541 raft_consensus.cc:482] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:18.664816 6541 raft_consensus.cc:504] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13684f39a48943529863a9a727f034d5" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46195 } } peers { permanent_uuid: "b41a27da1ced4afc841f9c3c86ea88e1" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 37229 } } peers { permanent_uuid: "58e07e1fcdf84c278fe56050c5e2d53c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46193 } }
I20260808 07:09:18.665036 6541 leader_election.cc:288] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195), b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229)
W20260808 07:09:18.665441 6000 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b41a27da1ced4afc841f9c3c86ea88e1 (127.18.63.66:37229): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
W20260808 07:09:18.665534 6001 leader_election.cc:341] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13684f39a48943529863a9a727f034d5 (127.18.63.67:46195): Not found: Tablet not found: f2d0979fe1964b9fba8fa5b784dd7166
I20260808 07:09:18.665650 6001 leader_election.cc:302] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58e07e1fcdf84c278fe56050c5e2d53c; no voters: 13684f39a48943529863a9a727f034d5, b41a27da1ced4afc841f9c3c86ea88e1
I20260808 07:09:18.665752 6541 raft_consensus.cc:2726] T f2d0979fe1964b9fba8fa5b784dd7166 P 58e07e1fcdf84c278fe56050c5e2d53c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:09:18.740998 6542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:18.740732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:46703)
0808 07:09:18.740887 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:18.740946 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260808 07:09:19.580982 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:19.580757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:39769)
0808 07:09:19.580894 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:19.580933 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260808 07:09:19.615852 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:19.615638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:45585)
0808 07:09:19.615766 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:19.615804 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260808 07:09:19.741784 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:19.741560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:41673)
0808 07:09:19.741697 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:19.741736 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260808 07:09:20.581806 6548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:20.581540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:48553)
0808 07:09:20.581715 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:20.581757 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260808 07:09:20.616660 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:20.616375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:51469)
0808 07:09:20.616567 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:20.616607 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260808 07:09:20.742518 6551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:20.742291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:56001)
0808 07:09:20.742428 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:20.742468 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260808 07:09:21.582502 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:21.582274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:34655)
0808 07:09:21.582412 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:21.582453 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260808 07:09:21.617391 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:21.617120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:38571)
0808 07:09:21.617250 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:21.617289 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260808 07:09:21.743211 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:21.742969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:54071)
0808 07:09:21.743119 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:21.743161 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260808 07:09:22.583323 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:22.583100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.65:39257)
0808 07:09:22.583234 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:22.583272 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260808 07:09:22.618095 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:22.617875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.66:53175)
0808 07:09:22.618008 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:22.618048 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 07:09:22.744091 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:22.743827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33635 (local address 127.18.63.67:58489)
0808 07:09:22.744002 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:22.744043 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260808 07:09:22.746891 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 5940
I20260808 07:09:22.756915 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 6149
I20260808 07:09:22.766172 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 6322
2026-08-08T07:09:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18887 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:09:22 dist-test-slave-bw88 krb5kdc[6562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:09:22 dist-test-slave-bw88 krb5kdc[6562](info): set up 2 sockets
Aug 08 07:09:22 dist-test-slave-bw88 krb5kdc[6562](info): commencing operation
krb5kdc: starting...
W20260808 07:09:23.963129 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:09:24.086342 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:24.185474 6579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:24.185809 6579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:24.185880 6579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:24.187813 6579 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:09:24.187882 6579 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:09:24.187902 6579 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:09:24.187919 6579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:24.187943 6579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:24.187964 6579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:24.187988 6579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:24.188011 6579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:24.190495 6579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:24.190810 6579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:24.191113 6579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:24.198318 6588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.198329 6585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.198586 6586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:24.198638 6579 server_base.cc:1034] running on GCE node
I20260808 07:09:24.199070 6579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:24.199498 6579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:24.200675 6579 hybrid_clock.cc:648] HybridClock initialized: now 1786172964200647 us; error 42 us; skew 500 ppm
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:24.204632 6579 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:24.205232 6579 webserver.cc:460] Webserver started at http://127.18.63.126:35581/ using document root <none> and password file <none>
I20260808 07:09:24.205571 6579 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:24.205648 6579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:24.205806 6579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:24.207613 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "eace4f6291964bf9985ead84a85ad775"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "2fa59325b1726c0be859b4bcff1f4d3d"
server_key_iv: "d4a37b16efff26d139504dbb1ab3dedb"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.208129 6579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "eace4f6291964bf9985ead84a85ad775"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "2fa59325b1726c0be859b4bcff1f4d3d"
server_key_iv: "d4a37b16efff26d139504dbb1ab3dedb"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.210139 6579 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 07:09:24.211259 6595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.211499 6579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:24.211687 6579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "eace4f6291964bf9985ead84a85ad775"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "2fa59325b1726c0be859b4bcff1f4d3d"
server_key_iv: "d4a37b16efff26d139504dbb1ab3dedb"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.211769 6579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:24.256521 6579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:24.259078 6579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:24.259245 6579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:24.263578 6579 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43733
I20260808 07:09:24.263623 6658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43733 every 8 connection(s)
I20260808 07:09:24.264012 6579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:24.264940 6659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:24.267310 6659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap starting.
I20260808 07:09:24.267907 6659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:24.268219 6659 log.cc:824] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:24.268944 6659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: No bootstrap required, opened a new log
I20260808 07:09:24.270607 6659 raft_consensus.cc:348] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:24.270746 6659 raft_consensus.cc:374] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:24.270768 6659 raft_consensus.cc:729] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Initialized, Role: FOLLOWER
I20260808 07:09:24.270964 6659 consensus_queue.cc:260] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:24.271049 6659 raft_consensus.cc:388] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:24.271107 6659 raft_consensus.cc:482] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:24.271157 6659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:24.271704 6659 raft_consensus.cc:504] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:24.271821 6659 leader_election.cc:302] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eace4f6291964bf9985ead84a85ad775; no voters:
I20260808 07:09:24.272017 6659 leader_election.cc:288] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:09:24.272061 6663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:24.272255 6663 raft_consensus.cc:686] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 LEADER]: Becoming Leader. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Running, Role: LEADER
I20260808 07:09:24.272284 6659 sys_catalog.cc:564] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:24.272347 6663 consensus_queue.cc:237] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:24.272944 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 6579
I20260808 07:09:24.273056 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:09:24.273272 18685 external_mini_cluster.cc:1428] Setting key 058fb90f9b584621c2739e96d5356717
I20260808 07:09:24.273896 6665 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eace4f6291964bf9985ead84a85ad775. Latest consensus state: current_term: 1 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:09:24.273989 6665 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:24.274233 6684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:24.274324 6664 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:09:24.274449 6664 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:24.274861 6684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:24.276456 6684 catalog_manager.cc:1347] Generated new cluster ID: d1f4b8ac00f94ce1916ec31d2e3e083e
I20260808 07:09:24.276510 6684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:24.285161 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.273877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.18.63.126:43733)
0808 07:09:24.274052 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.274062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.274099 (+ 37us) server_negotiation.cc:407] Connection header received
0808 07:09:24.274224 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.274283 (+ 59us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.274385 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.274565 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:24.275349 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.276683 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.277807 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.278428 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.278545 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.281476 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:24.281480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:24.281481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:24.281493 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:24.283403 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.283831 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.283834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.283834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.283950 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.284184 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.284185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.284185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.284489 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:24.284617 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.284803 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.284942 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260808 07:09:24.287194 6684 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:09:24.287671 6684 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:24.290437 6684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Generated new TSK 0
I20260808 07:09:24.290616 6684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:09:24.323779 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:24.409771 6691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:24.409986 6691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:24.410012 6691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:24.411201 6691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:24.411244 6691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:24.411298 6691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:09:24.412771 6691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:24.413030 6691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:24.413275 6691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:24.418519 6705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.418530 6699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.418618 6703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:24.418802 6691 server_base.cc:1034] running on GCE node
I20260808 07:09:24.419008 6691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:24.419247 6691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:24.420466 6691 hybrid_clock.cc:648] HybridClock initialized: now 1786172964420442 us; error 30 us; skew 500 ppm
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:24.423280 6691 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:09:24.423861 6691 webserver.cc:460] Webserver started at http://127.18.63.65:35877/ using document root <none> and password file <none>
I20260808 07:09:24.424135 6691 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:24.424186 6691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:24.424386 6691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:24.425686 6691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "e855d76a032f4c19b4c065715b4821b5"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "77b1c86463233ec2e76870d7b8711033"
server_key_iv: "5d27b2359643b24ee6ec27c5b555ee10"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.426083 6691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "e855d76a032f4c19b4c065715b4821b5"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "77b1c86463233ec2e76870d7b8711033"
server_key_iv: "5d27b2359643b24ee6ec27c5b555ee10"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.427603 6691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:24.428417 6714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.428798 6691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:24.428928 6691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "e855d76a032f4c19b4c065715b4821b5"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "77b1c86463233ec2e76870d7b8711033"
server_key_iv: "5d27b2359643b24ee6ec27c5b555ee10"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.429016 6691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:24.441776 6691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:24.443738 6691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:24.443922 6691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:24.444173 6691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:24.444581 6691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:24.444648 6691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.444695 6691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:24.444726 6691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.451538 6691 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:45595
I20260808 07:09:24.451602 6853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:45595 every 8 connection(s)
I20260808 07:09:24.451968 6691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:24.459753 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 6691
I20260808 07:09:24.459841 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:09:24.460091 18685 external_mini_cluster.cc:1428] Setting key 5d9be24e490914e8cd425afd925b3a19
I20260808 07:09:24.461447 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.452751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:60489 (local address 127.18.63.126:43733)
0808 07:09:24.452843 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.452846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.453019 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:09:24.453547 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.453548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.453563 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.453610 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:24.454519 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.455003 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.455964 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.456180 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.456237 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.458522 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:24.458524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:24.458524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:24.458534 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:24.459943 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.460417 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.460418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.460419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.460483 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.460755 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.460756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.460757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.460874 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:24.460945 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.461244 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.461332 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:09:24.461658 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.452711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:60489)
0808 07:09:24.452901 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.452936 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.453455 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.453663 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.453666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.454222 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:24.454406 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.454411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.455092 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.455093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.455881 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.455883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.456258 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.456259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.456359 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.456818 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:24.456826 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:24.458390 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:24.460020 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.460029 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.460031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.460329 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.460556 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.460557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.460557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.460681 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.460949 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:24.460956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:24.461088 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.461220 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.461408 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260808 07:09:24.462235 6854 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:24.462328 6854 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:24.462514 6854 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:24.463050 6618 ts_manager.cc:194] Registered new tserver with Master: e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595)
I20260808 07:09:24.463673 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:60489
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:09:24.488268 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:24.576035 6863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:24.576236 6863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:24.576262 6863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:24.577430 6863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:24.577473 6863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:24.577499 6863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:09:24.578948 6863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:24.579138 6863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:24.579317 6863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:24.584656 6869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.584715 6871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.584656 6868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:24.584709 6863 server_base.cc:1034] running on GCE node
I20260808 07:09:24.585124 6863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:24.585431 6863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:24.586675 6863 hybrid_clock.cc:648] HybridClock initialized: now 1786172964586655 us; error 30 us; skew 500 ppm
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:24.589471 6863 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:09:24.589915 6863 webserver.cc:460] Webserver started at http://127.18.63.66:36573/ using document root <none> and password file <none>
I20260808 07:09:24.590162 6863 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:24.590233 6863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:24.590376 6863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:24.591475 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "57209da1fc4f4797b11f20b50282bdcd"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "8d51687d20c3b544943ef18fabae0297"
server_key_iv: "8dfd88a5f174e2b0db25d822969470a0"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.591849 6863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "57209da1fc4f4797b11f20b50282bdcd"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "8d51687d20c3b544943ef18fabae0297"
server_key_iv: "8dfd88a5f174e2b0db25d822969470a0"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.593297 6863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:09:24.594130 6879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.594300 6863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:24.594411 6863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "57209da1fc4f4797b11f20b50282bdcd"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "8d51687d20c3b544943ef18fabae0297"
server_key_iv: "8dfd88a5f174e2b0db25d822969470a0"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.594527 6863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:24.603135 6863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:24.605119 6863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:24.605280 6863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:24.605541 6863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:24.605840 6863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:24.605895 6863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.605943 6863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:24.605990 6863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.612337 6863 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:32857
I20260808 07:09:24.612391 7017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:32857 every 8 connection(s)
I20260808 07:09:24.612700 6863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:09:24.615003 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 6863
I20260808 07:09:24.615100 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:09:24.615341 18685 external_mini_cluster.cc:1428] Setting key a77b42570ae99f6ebe14dba5818428bd
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:24.623658 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.613443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:38303 (local address 127.18.63.126:43733)
0808 07:09:24.613490 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.613492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.613711 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:09:24.614246 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.614248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.614262 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.614314 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:24.615451 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.616142 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.617424 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.617736 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.617811 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.620546 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:24.620548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:24.620548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:24.620557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:24.622238 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.622644 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.622646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.622646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.622716 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.622957 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.622958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.622959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.623123 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:24.623206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.623457 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.623549 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:09:24.623695 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.613425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:38303)
0808 07:09:24.613591 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.613625 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.614140 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.614397 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.614401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.615025 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:24.615326 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.615333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.616264 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.616265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.617324 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.617328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.617832 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.617833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.617949 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.618506 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:24.618518 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:24.620446 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:24.622327 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.622334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.622335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.622548 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.622789 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.622789 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.622790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.622888 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.623198 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:24.623202 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:24.623315 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.623397 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.623516 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260808 07:09:24.624284 7019 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:24.624384 7019 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:24.624560 7019 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:24.625032 6618 ts_manager.cc:194] Registered new tserver with Master: 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857)
I20260808 07:09:24.625464 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:38303
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:09:24.645949 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:24.732585 7026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:24.732810 7026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:24.732851 7026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:24.734077 7026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:24.734141 7026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:24.734185 7026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:09:24.735685 7026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:24.735961 7026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:24.736231 7026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:09:24.741624 7026 server_base.cc:1034] running on GCE node
W20260808 07:09:24.741586 7033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.741621 7035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:24.741593 7032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:24.742106 7026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:24.742360 7026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:24.743530 7026 hybrid_clock.cc:648] HybridClock initialized: now 1786172964743512 us; error 41 us; skew 500 ppm
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:24.746254 7026 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:09:24.746691 7026 webserver.cc:460] Webserver started at http://127.18.63.67:43457/ using document root <none> and password file <none>
I20260808 07:09:24.746945 7026 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:24.747016 7026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:24.747169 7026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:09:24.748204 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "20f9257212bf458a90bca526a5c9f5c2"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "e2b6db2389622e834131590217d029f8"
server_key_iv: "ca3cdf85e3d0ad3a91f4418278b16576"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.748587 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "20f9257212bf458a90bca526a5c9f5c2"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "e2b6db2389622e834131590217d029f8"
server_key_iv: "ca3cdf85e3d0ad3a91f4418278b16576"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.750027 7026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260808 07:09:24.750881 7044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.751103 7026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:24.751230 7026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "20f9257212bf458a90bca526a5c9f5c2"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "e2b6db2389622e834131590217d029f8"
server_key_iv: "ca3cdf85e3d0ad3a91f4418278b16576"
server_key_version: "encryptionkey@0"
I20260808 07:09:24.751307 7026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:24.769424 7026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:24.771796 7026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:24.771961 7026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:24.772248 7026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:24.772616 7026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:24.772675 7026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.772732 7026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:24.772779 7026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:24.779263 7026 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:33999
I20260808 07:09:24.779312 7190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:33999 every 8 connection(s)
I20260808 07:09:24.779619 7026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:09:24.781876 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7026
I20260808 07:09:24.782012 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:09:24.782231 18685 external_mini_cluster.cc:1428] Setting key c89cf109a34804a96b1b73283dfa03d2
Aug 08 07:09:24 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:24.789459 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.780303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:35599 (local address 127.18.63.126:43733)
0808 07:09:24.780387 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.780390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.780640 (+ 250us) server_negotiation.cc:407] Connection header received
0808 07:09:24.781207 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.781209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.781223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.781303 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:24.782462 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.783225 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.784235 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.784450 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.784529 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.786788 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:24.786789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:24.786790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:24.786797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:24.787986 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.788414 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.788416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.788416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.788483 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.788748 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.788749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.788750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.788869 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:24.788943 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.789233 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.789314 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 07:09:24.789644 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.780278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:35599)
0808 07:09:24.780475 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.780511 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.781063 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.781419 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.781423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.782087 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:24.782347 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.782354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.783315 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.783317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.784143 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.784145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.784538 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.784539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.784693 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.785189 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:24.785201 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:24.786687 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:24.788098 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.788107 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.788109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.788333 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.788552 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:24.788553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:24.788553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:24.788670 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:24.788949 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:24.788956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:24.789094 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.789208 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.789395 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260808 07:09:24.790148 7193 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:24.790253 7193 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:24.790462 7193 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:24.790910 6618 ts_manager.cc:194] Registered new tserver with Master: 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
I20260808 07:09:24.791278 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:35599
I20260808 07:09:24.793996 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:09:24.801084 6681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.795344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.18.63.126:43733)
0808 07:09:24.795406 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.795408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.795541 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:09:24.795580 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.795580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.795589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.795628 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:24.796154 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.796598 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.797417 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.797606 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.797669 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.798894 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:24.798895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:24.798895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:24.798900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:24.800050 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.800384 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.800385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.800385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.800436 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:24.800606 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:24.800606 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:24.800607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:24.800738 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:24.800810 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.800911 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.800968 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:09:24.802903 6618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:09:24.804476 6618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:09:24.811460 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:32857 (local address 127.0.0.1:39268)
0808 07:09:24.807446 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.807493 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.807574 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.808396 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.808397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.808399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:24.808434 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.808435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809285 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.809286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.810436 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.810439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.811170 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.811171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.811250 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.811262 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.811335 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.811385 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":114,"thread_start_us":69,"threads_started":1}
I20260808 07:09:24.811468 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.18.63.66:32857)
0808 07:09:24.808229 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.808233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.808242 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:09:24.808278 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.808280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.808293 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.808381 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:24.808488 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809185 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.810533 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.811064 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.811165 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.811233 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.811350 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.811390 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":129,"threads_started":1}
I20260808 07:09:24.812242 6943 tablet_service.cc:1467] Processing CreateTablet for tablet 450538e50e8b417d9051c335bf652e37 (DEFAULT_TABLE table=test-table [id=bac72afa7f174f4f93c363c981e37925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:24.812424 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33999 (local address 127.0.0.1:48814)
0808 07:09:24.807691 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.807701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.807743 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.808082 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.808085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.808088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:24.808157 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.808159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809709 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.809709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.811124 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.811126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.812161 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.812162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.812247 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.812252 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.812288 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.812336 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":80,"thread_start_us":92,"threads_started":1}
I20260808 07:09:24.812649 6943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450538e50e8b417d9051c335bf652e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:24.812893 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.18.63.67:33999)
0808 07:09:24.807896 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.807901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.807917 (+ 16us) server_negotiation.cc:407] Connection header received
0808 07:09:24.807953 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.807955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.807969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.808046 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:24.808251 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809567 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.811221 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.812043 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.812627 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.812694 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.812776 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.812821 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260808 07:09:24.812932 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.18.63.65:45595)
0808 07:09:24.807914 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.807921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.807938 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:09:24.807971 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.807973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.807985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.808090 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:24.808331 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809577 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.811220 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.812248 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.812325 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.812382 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.812807 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.812856 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":141,"threads_started":1}
I20260808 07:09:24.813262 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.807154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:45595 (local address 127.0.0.1:34032)
0808 07:09:24.807478 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.807500 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.807574 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.808109 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.808110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.808113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:24.808223 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.808225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.809702 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.809703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.811102 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.811106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.812338 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.812340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.812401 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.812405 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.812737 (+ 332us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.813196 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":92,"thread_start_us":102,"threads_started":1}
I20260808 07:09:24.813736 7108 tablet_service.cc:1467] Processing CreateTablet for tablet 450538e50e8b417d9051c335bf652e37 (DEFAULT_TABLE table=test-table [id=bac72afa7f174f4f93c363c981e37925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:24.814122 7108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450538e50e8b417d9051c335bf652e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:24.814060 6774 tablet_service.cc:1467] Processing CreateTablet for tablet 450538e50e8b417d9051c335bf652e37 (DEFAULT_TABLE table=test-table [id=bac72afa7f174f4f93c363c981e37925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:24.814384 6774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 450538e50e8b417d9051c335bf652e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:24.815745 7226 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Bootstrap starting.
I20260808 07:09:24.816432 7226 tablet_bootstrap.cc:654] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:24.816880 7226 log.cc:824] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:24.817828 7226 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: No bootstrap required, opened a new log
I20260808 07:09:24.817915 7226 ts_tablet_manager.cc:1397] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:09:24.818238 7231 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Bootstrap starting.
I20260808 07:09:24.818454 7230 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Bootstrap starting.
I20260808 07:09:24.819125 7230 tablet_bootstrap.cc:654] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:24.819715 7230 log.cc:824] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:24.820222 7226 raft_consensus.cc:348] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.820415 7226 raft_consensus.cc:374] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:24.820461 7226 raft_consensus.cc:729] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57209da1fc4f4797b11f20b50282bdcd, State: Initialized, Role: FOLLOWER
I20260808 07:09:24.820564 7231 tablet_bootstrap.cc:654] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:24.820595 7226 consensus_queue.cc:260] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.820891 7230 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: No bootstrap required, opened a new log
I20260808 07:09:24.820952 7226 ts_tablet_manager.cc:1428] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:09:24.821017 7019 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
I20260808 07:09:24.821002 7230 ts_tablet_manager.cc:1397] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:09:24.821209 7231 log.cc:824] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:24.822068 7231 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: No bootstrap required, opened a new log
I20260808 07:09:24.822155 7231 ts_tablet_manager.cc:1397] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 07:09:24.823283 7230 raft_consensus.cc:348] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.823428 7230 raft_consensus.cc:374] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:24.823452 7230 raft_consensus.cc:729] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e855d76a032f4c19b4c065715b4821b5, State: Initialized, Role: FOLLOWER
I20260808 07:09:24.823585 7230 consensus_queue.cc:260] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.823614 7231 raft_consensus.cc:348] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.823822 7231 raft_consensus.cc:374] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:24.823865 7231 raft_consensus.cc:729] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f9257212bf458a90bca526a5c9f5c2, State: Initialized, Role: FOLLOWER
I20260808 07:09:24.823904 7230 ts_tablet_manager.cc:1428] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:09:24.823949 6854 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
I20260808 07:09:24.823982 7231 consensus_queue.cc:260] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.824254 7231 ts_tablet_manager.cc:1428] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:09:24.824307 7193 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
W20260808 07:09:24.863317 7020 tablet.cc:2369] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:09:24.907338 7238 raft_consensus.cc:482] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:24.907548 7238 raft_consensus.cc:504] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.907960 7238 leader_election.cc:288] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
I20260808 07:09:24.911113 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.908120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:32857 (local address 127.18.63.65:33263)
0808 07:09:24.908184 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.908198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.908246 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.908427 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.908428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.908430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:24.908548 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.908550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.909378 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.909379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.910184 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.910187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.910842 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.910935 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.910938 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.910981 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.911016 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 07:09:24.911137 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.908218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:33263 (local address 127.18.63.66:32857)
0808 07:09:24.908277 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.908280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.908291 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:09:24.908326 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.908327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.908340 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.908415 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:24.908692 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.909255 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910287 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.910749 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910812 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.910820 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.911015 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.911063 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:09:24.911199 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.908280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:56855 (local address 127.18.63.67:33999)
0808 07:09:24.908347 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.908349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.908483 (+ 134us) server_negotiation.cc:407] Connection header received
0808 07:09:24.908540 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.908541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.908552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.908622 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:24.908854 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.909348 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910384 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.910778 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910879 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.910887 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.911079 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.911126 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:09:24.911258 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.908151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33999 (local address 127.18.63.65:56855)
0808 07:09:24.908399 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.908409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:24.908469 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:24.908689 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:24.908691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:24.908693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:24.908768 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.908769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.909471 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.909472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.910295 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:24.910297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.910904 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.910904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:24.910987 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.910992 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:09:24.911053 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:09:24.911090 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":12,"thread_start_us":156,"threads_started":1}
I20260808 07:09:24.911538 6964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450538e50e8b417d9051c335bf652e37" candidate_uuid: "e855d76a032f4c19b4c065715b4821b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57209da1fc4f4797b11f20b50282bdcd" is_pre_election: true
I20260808 07:09:24.911574 7135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450538e50e8b417d9051c335bf652e37" candidate_uuid: "e855d76a032f4c19b4c065715b4821b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f9257212bf458a90bca526a5c9f5c2" is_pre_election: true
I20260808 07:09:24.911736 6964 raft_consensus.cc:2454] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e855d76a032f4c19b4c065715b4821b5 in term 0.
I20260808 07:09:24.911734 7135 raft_consensus.cc:2454] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e855d76a032f4c19b4c065715b4821b5 in term 0.
I20260808 07:09:24.912009 6719 leader_election.cc:302] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57209da1fc4f4797b11f20b50282bdcd, e855d76a032f4c19b4c065715b4821b5; no voters:
I20260808 07:09:24.912200 7238 raft_consensus.cc:2781] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:09:24.912243 7238 raft_consensus.cc:482] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:09:24.912259 7238 raft_consensus.cc:3037] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:24.912981 7238 raft_consensus.cc:504] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.913136 7238 leader_election.cc:288] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 1 election: Requested vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
I20260808 07:09:24.913295 6964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450538e50e8b417d9051c335bf652e37" candidate_uuid: "e855d76a032f4c19b4c065715b4821b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57209da1fc4f4797b11f20b50282bdcd"
I20260808 07:09:24.913324 7135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "450538e50e8b417d9051c335bf652e37" candidate_uuid: "e855d76a032f4c19b4c065715b4821b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f9257212bf458a90bca526a5c9f5c2"
I20260808 07:09:24.913416 7135 raft_consensus.cc:3037] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:24.913412 6964 raft_consensus.cc:3037] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:24.914371 7135 raft_consensus.cc:2454] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e855d76a032f4c19b4c065715b4821b5 in term 1.
I20260808 07:09:24.914371 6964 raft_consensus.cc:2454] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e855d76a032f4c19b4c065715b4821b5 in term 1.
I20260808 07:09:24.914556 6719 leader_election.cc:302] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57209da1fc4f4797b11f20b50282bdcd, e855d76a032f4c19b4c065715b4821b5; no voters:
I20260808 07:09:24.914655 7238 raft_consensus.cc:2781] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:24.914830 7238 raft_consensus.cc:686] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 LEADER]: Becoming Leader. State: Replica: e855d76a032f4c19b4c065715b4821b5, State: Running, Role: LEADER
I20260808 07:09:24.914974 7238 consensus_queue.cc:237] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:24.915956 6617 catalog_manager.cc:5495] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e855d76a032f4c19b4c065715b4821b5 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "e855d76a032f4c19b4c065715b4821b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } health_report { overall_health: UNKNOWN } } }
I20260808 07:09:24.926412 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.924209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.18.63.65:45595)
0808 07:09:24.924266 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:24.924273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:24.924281 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:09:24.924310 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:24.924310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:24.924326 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:24.924384 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:24.924488 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.924930 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.925742 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:24.925920 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:24.925974 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:24.926103 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:24.926197 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:09:24.926255 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:24.926287 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 07:09:24.926303 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:09:24.928237 6964 raft_consensus.cc:1261] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Refusing update from remote peer e855d76a032f4c19b4c065715b4821b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:24.928237 7135 raft_consensus.cc:1261] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Refusing update from remote peer e855d76a032f4c19b4c065715b4821b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:24.928650 7241 consensus_queue.cc:1035] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:24.928786 7238 consensus_queue.cc:1035] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:24.932165 7246 mvcc.cc:205] Tried to move back new op lower bound from 7316164464343240704 to 7316164464292372480. Current Snapshot: MvccSnapshot[applied={T|T < 7316164464343240704}]
I20260808 07:09:24.932324 7245 mvcc.cc:205] Tried to move back new op lower bound from 7316164464343240704 to 7316164464292372480. Current Snapshot: MvccSnapshot[applied={T|T < 7316164464343240704 or (T in {7316164464343240704})}]
I20260808 07:09:24.933568 6617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:24.933672 6617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:24.934439 6617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Sending DeleteTablet for 3 replicas of tablet 450538e50e8b417d9051c335bf652e37
I20260808 07:09:24.934791 6774 tablet_service.cc:1514] Processing DeleteTablet for tablet 450538e50e8b417d9051c335bf652e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:24 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:34032
I20260808 07:09:24.934824 7108 tablet_service.cc:1514] Processing DeleteTablet for tablet 450538e50e8b417d9051c335bf652e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:24 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:48814
I20260808 07:09:24.934875 6943 tablet_service.cc:1514] Processing DeleteTablet for tablet 450538e50e8b417d9051c335bf652e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:24 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:39268
I20260808 07:09:24.934937 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 6691
I20260808 07:09:24.935115 7257 tablet_replica.cc:330] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: stopping tablet replica
I20260808 07:09:24.935221 7257 raft_consensus.cc:2229] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:24.935324 7257 raft_consensus.cc:2258] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:24.938117 7258 tablet_replica.cc:330] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: stopping tablet replica
I20260808 07:09:24.938259 7258 raft_consensus.cc:2229] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:24.938390 7258 raft_consensus.cc:2258] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:24.939355 7257 ts_tablet_manager.cc:1905] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:24.939406 7258 ts_tablet_manager.cc:1905] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:24.941048 7257 ts_tablet_manager.cc:1918] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:24.941179 7257 log.cc:1196] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/450538e50e8b417d9051c335bf652e37
I20260808 07:09:24.941424 7258 ts_tablet_manager.cc:1918] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:24.941506 7258 log.cc:1196] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/450538e50e8b417d9051c335bf652e37
I20260808 07:09:24.941537 7257 ts_tablet_manager.cc:1939] T 450538e50e8b417d9051c335bf652e37 P 20f9257212bf458a90bca526a5c9f5c2: Deleting consensus metadata
I20260808 07:09:24.941808 7258 ts_tablet_manager.cc:1939] T 450538e50e8b417d9051c335bf652e37 P 57209da1fc4f4797b11f20b50282bdcd: Deleting consensus metadata
I20260808 07:09:24.942013 6600 catalog_manager.cc:4841] TS 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): tablet 450538e50e8b417d9051c335bf652e37 (table test-table [id=bac72afa7f174f4f93c363c981e37925]) successfully deleted
I20260808 07:09:24.942299 6601 catalog_manager.cc:4841] TS 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): tablet 450538e50e8b417d9051c335bf652e37 (table test-table [id=bac72afa7f174f4f93c363c981e37925]) successfully deleted
W20260808 07:09:24.948956 6597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:45595 (error 108)
I20260808 07:09:24.949169 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 6863
I20260808 07:09:24.951372 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:24.950552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:45595 (local address 127.0.0.1:34050)
0808 07:09:24.951281 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:24.951327 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:45595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":705}
W20260808 07:09:24.951756 6597 catalog_manager.cc:4584] TS e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 450538e50e8b417d9051c335bf652e37: Network error: Client connection negotiation failed: client connection to 127.18.63.65:45595: connect: Connection refused (error 111)
I20260808 07:09:24.957621 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 7026
I20260808 07:09:24.969242 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 6579
I20260808 07:09:44.978284 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--webserver_interface=127.18.63.126
--webserver_port=35581
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:09:45.061997 7260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:45.062213 7260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:45.062235 7260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:45.063408 7260 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:09:45.063452 7260 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:09:45.063464 7260 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:09:45.063477 7260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:09:45.063491 7260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:45.063548 7260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:45.063565 7260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:09:45.063588 7260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:09:45.065084 7260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=35581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:45.065368 7260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:45.065615 7260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:45.075067 7270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.075119 7268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.075119 7267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.075156 7260 server_base.cc:1034] running on GCE node
I20260808 07:09:45.075534 7260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:45.075835 7260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:45.076992 7260 hybrid_clock.cc:648] HybridClock initialized: now 1786172985076979 us; error 44 us; skew 500 ppm
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:45.079434 7260 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:09:45.079897 7260 webserver.cc:460] Webserver started at http://127.18.63.126:35581/ using document root <none> and password file <none>
I20260808 07:09:45.080170 7260 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:45.080252 7260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:45.081635 7260 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:45.082329 7277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.082508 7260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:45.082612 7260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "eace4f6291964bf9985ead84a85ad775"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "2fa59325b1726c0be859b4bcff1f4d3d"
server_key_iv: "d4a37b16efff26d139504dbb1ab3dedb"
server_key_version: "encryptionkey@0"
I20260808 07:09:45.083050 7260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:45.108014 7260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:45.110252 7260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:45.110420 7260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:45.114861 7260 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43733
I20260808 07:09:45.114900 7343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43733 every 8 connection(s)
I20260808 07:09:45.115324 7260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:09:45.116849 7344 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:09:45.117635 7344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap starting.
I20260808 07:09:45.124780 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7260
I20260808 07:09:45.125159 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:45595
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=35877
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:45.127467 7344 log.cc:824] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:45.129806 7344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:45.130133 7344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap complete.
I20260808 07:09:45.132483 7344 raft_consensus.cc:348] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:45.132717 7344 raft_consensus.cc:729] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Initialized, Role: FOLLOWER
I20260808 07:09:45.132889 7344 consensus_queue.cc:260] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:45.132951 7344 raft_consensus.cc:388] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:09:45.133014 7344 raft_consensus.cc:482] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:09:45.133064 7344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 1 FOLLOWER]: Advancing to term 2
I20260808 07:09:45.134161 7344 raft_consensus.cc:504] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:45.134251 7344 leader_election.cc:302] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eace4f6291964bf9985ead84a85ad775; no voters:
I20260808 07:09:45.134429 7344 leader_election.cc:288] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 07:09:45.134472 7350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 07:09:45.134606 7350 raft_consensus.cc:686] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 LEADER]: Becoming Leader. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Running, Role: LEADER
I20260808 07:09:45.134717 7344 sys_catalog.cc:564] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:09:45.134696 7350 consensus_queue.cc:237] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:09:45.135021 7350 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eace4f6291964bf9985ead84a85ad775. Latest consensus state: current_term: 2 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:09:45.135095 7350 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:45.135264 7352 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:09:45.135336 7352 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:09:45.135613 7357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:09:45.136530 7357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:09:45.136672 7357 catalog_manager.cc:1259] Loaded cluster ID: d1f4b8ac00f94ce1916ec31d2e3e083e
I20260808 07:09:45.136724 7357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:09:45.138237 7357 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:09:45.138418 7357 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Loaded TSK: 0
I20260808 07:09:45.152262 7357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Generated new TSK 1
I20260808 07:09:45.152351 7357 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:09:45.216212 7346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:45.216418 7346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:45.216441 7346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:45.217597 7346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:45.217638 7346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:45.217664 7346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:09:45.219084 7346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:45595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=35877
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:45.219338 7346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:45.219571 7346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:45.225996 7390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.226033 7386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.226082 7346 server_base.cc:1034] running on GCE node
W20260808 07:09:45.226228 7385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.226459 7346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:45.226729 7346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:45.227886 7346 hybrid_clock.cc:648] HybridClock initialized: now 1786172985227869 us; error 34 us; skew 500 ppm
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:45.230690 7346 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:09:45.231236 7346 webserver.cc:460] Webserver started at http://127.18.63.65:35877/ using document root <none> and password file <none>
I20260808 07:09:45.231541 7346 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:45.231628 7346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:45.233400 7346 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:09:45.234217 7399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.234402 7346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:45.234508 7346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "e855d76a032f4c19b4c065715b4821b5"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "77b1c86463233ec2e76870d7b8711033"
server_key_iv: "5d27b2359643b24ee6ec27c5b555ee10"
server_key_version: "encryptionkey@0"
I20260808 07:09:45.234946 7346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:45.267190 7346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:45.269640 7346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:45.269837 7346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:45.270128 7346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:45.270654 7407 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:09:45.272344 7346 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:09:45.272406 7346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:09:45.272441 7346 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:09:45.273300 7346 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:09:45.273388 7346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:45.273442 7407 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Bootstrap starting.
I20260808 07:09:45.280284 7346 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:45595
I20260808 07:09:45.280351 7545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:45595 every 8 connection(s)
I20260808 07:09:45.280669 7346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:09:45.281960 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7346
I20260808 07:09:45.282344 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:32857
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=36573
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:45.285699 7407 log.cc:824] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:45.289103 7407 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:09:45.289500 7407 tablet_bootstrap.cc:492] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Bootstrap complete.
I20260808 07:09:45.291064 7407 ts_tablet_manager.cc:1397] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:45.292748 7407 raft_consensus.cc:348] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:45.293169 7407 raft_consensus.cc:729] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e855d76a032f4c19b4c065715b4821b5, State: Initialized, Role: FOLLOWER
I20260808 07:09:45.293325 7407 consensus_queue.cc:260] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } }
I20260808 07:09:45.293676 7407 ts_tablet_manager.cc:1428] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 07:09:45.295789 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.281490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:36193 (local address 127.18.63.126:43733)
0808 07:09:45.285510 (+ 4020us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.285521 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.285572 (+ 51us) server_negotiation.cc:407] Connection header received
0808 07:09:45.285719 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.285726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.285788 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.285950 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.286844 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.287811 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.288900 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.289433 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.289561 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.292058 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.292062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.292062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.292072 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.294032 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.294452 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.294455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.294456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.294593 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.294864 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.294865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.294865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.295089 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.295169 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.295466 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.295599 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3950,"thread_start_us":99,"threads_started":1}
I20260808 07:09:45.295789 7549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.281399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:36193)
0808 07:09:45.281596 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.281640 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.282124 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.286006 (+ 3882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.286011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.286405 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:45.286705 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.286713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.287939 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.287941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.288810 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.288812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.289599 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.289601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.289730 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.290298 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:45.290306 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:45.291940 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:45.294128 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.294136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.294138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.294364 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.294671 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.294672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.294673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.294792 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.295166 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:45.295173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:45.295301 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.295399 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.295575 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260808 07:09:45.296589 7546 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:45.296689 7546 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:45.296871 7546 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:45.297626 7299 ts_manager.cc:194] Registered new tserver with Master: e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595)
I20260808 07:09:45.298831 7299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:36193
I20260808 07:09:45.299798 7546 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.65@KRBTEST.COM
I20260808 07:09:45.306798 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.298351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.18.63.65:45595)
0808 07:09:45.299151 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.299155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.299166 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:09:45.299199 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.299201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.299217 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.299290 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.299712 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.300619 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.301438 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.301737 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.301829 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.303884 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.303891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.303892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.303898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.305481 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.305922 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.305925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.305925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.305980 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.306197 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.306198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.306199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.306355 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.306430 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.306630 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.306732 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":88,"threads_started":1}
I20260808 07:09:45.306821 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.298286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:45595 (local address 127.0.0.1:49144)
0808 07:09:45.298498 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.298526 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.298586 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.299355 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.299360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.299586 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:45.299630 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.299631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.300698 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.300699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.301294 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.301296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.301832 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.301833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.301917 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.302382 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:45.302387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:45.303781 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:45.305559 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.305566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.305568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.305834 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.306040 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.306041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.306041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.306110 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.306428 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:45.306435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:45.306486 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.306578 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.306717 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
I20260808 07:09:45.307328 7458 tablet_service.cc:1514] Processing DeleteTablet for tablet 450538e50e8b417d9051c335bf652e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:24 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49144
I20260808 07:09:45.307953 7561 tablet_replica.cc:330] stopping tablet replica
I20260808 07:09:45.308081 7561 raft_consensus.cc:2229] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:45.308169 7561 raft_consensus.cc:2258] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:45.308477 7561 ts_tablet_manager.cc:1905] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:45.310894 7561 ts_tablet_manager.cc:1918] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:45.310966 7561 log.cc:1196] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal/wals/450538e50e8b417d9051c335bf652e37
I20260808 07:09:45.311239 7561 ts_tablet_manager.cc:1939] T 450538e50e8b417d9051c335bf652e37 P e855d76a032f4c19b4c065715b4821b5: Deleting consensus metadata
I20260808 07:09:45.311776 7278 catalog_manager.cc:4841] TS e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595): tablet 450538e50e8b417d9051c335bf652e37 (table test-table [id=bac72afa7f174f4f93c363c981e37925]) successfully deleted
W20260808 07:09:45.372805 7553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:45.373001 7553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:45.373025 7553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:45.374416 7553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:45.374462 7553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:45.374490 7553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:09:45.375860 7553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=36573
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:45.376053 7553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:45.376221 7553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:09:45.381521 7571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.381520 7567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.381670 7553 server_base.cc:1034] running on GCE node
W20260808 07:09:45.381526 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.381925 7553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:45.382121 7553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:45.383258 7553 hybrid_clock.cc:648] HybridClock initialized: now 1786172985383243 us; error 30 us; skew 500 ppm
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:45.385795 7553 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:09:45.386262 7553 webserver.cc:460] Webserver started at http://127.18.63.66:36573/ using document root <none> and password file <none>
I20260808 07:09:45.386489 7553 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:45.386577 7553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:45.388010 7553 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:45.388576 7584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.388758 7553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.388859 7553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "57209da1fc4f4797b11f20b50282bdcd"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "8d51687d20c3b544943ef18fabae0297"
server_key_iv: "8dfd88a5f174e2b0db25d822969470a0"
server_key_version: "encryptionkey@0"
I20260808 07:09:45.389194 7553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:45.400715 7553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:45.402992 7553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:45.403152 7553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:45.403398 7553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:45.403707 7553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:45.403759 7553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.403800 7553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:45.403828 7553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.410588 7553 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:32857
I20260808 07:09:45.410629 7717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:32857 every 8 connection(s)
I20260808 07:09:45.410988 7553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:45.419589 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7553
I20260808 07:09:45.419942 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:33999
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=43457
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:09:45.420378 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.411731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:44535 (local address 127.18.63.126:43733)
0808 07:09:45.411803 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.411805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.412000 (+ 195us) server_negotiation.cc:407] Connection header received
0808 07:09:45.412492 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.412493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.412507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.412546 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.413569 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.414082 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.415087 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.415292 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.415344 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.417630 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.417632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.417632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.417639 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.418927 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.419330 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.419331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.419332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.419383 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.419688 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.419689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.419689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.419809 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.419862 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.420227 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.420311 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:09:45.420670 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.411722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:44535)
0808 07:09:45.411878 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.411915 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.412395 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.412608 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.412611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.413180 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:45.413465 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.413471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.414165 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.414165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.415010 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.415012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.415365 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.415365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.415465 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.416040 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:45.416049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:45.417526 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:45.419024 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.419031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.419033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.419248 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.419451 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.419452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.419452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.419598 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.419875 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:45.419882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:45.420070 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.420194 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.420392 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 07:09:45.421314 7718 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:45.421458 7718 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:45.421651 7718 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:45.422075 7299 ts_manager.cc:194] Registered new tserver with Master: 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857)
I20260808 07:09:45.422465 7299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:44535
W20260808 07:09:45.509409 7721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:09:45.509630 7721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:09:45.509654 7721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:09:45.510756 7721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:09:45.510789 7721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:09:45.510814 7721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:09:45.512157 7721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:33999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=43457
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:09:45.512353 7721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:09:45.512533 7721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:09:45.517755 7721 server_base.cc:1034] running on GCE node
W20260808 07:09:45.517776 7727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.517758 7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:09:45.517776 7728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:09:45.518294 7721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:09:45.518579 7721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:09:45.519757 7721 hybrid_clock.cc:648] HybridClock initialized: now 1786172985519743 us; error 46 us; skew 500 ppm
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:09:45.522341 7721 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:09:45.522783 7721 webserver.cc:460] Webserver started at http://127.18.63.67:43457/ using document root <none> and password file <none>
I20260808 07:09:45.523010 7721 fs_manager.cc:359] Metadata directory not provided
I20260808 07:09:45.523092 7721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:09:45.524413 7721 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:09:45.524984 7738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.525157 7721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:09:45.525260 7721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "20f9257212bf458a90bca526a5c9f5c2"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "e2b6db2389622e834131590217d029f8"
server_key_iv: "ca3cdf85e3d0ad3a91f4418278b16576"
server_key_version: "encryptionkey@0"
I20260808 07:09:45.525666 7721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:09:45.538303 7721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:09:45.540594 7721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:09:45.540761 7721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:09:45.541018 7721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:09:45.541420 7721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:09:45.541473 7721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.541513 7721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:09:45.541549 7721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:09:45.547744 7721 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:33999
I20260808 07:09:45.547799 7895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:33999 every 8 connection(s)
I20260808 07:09:45.548130 7721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
Aug 08 07:09:45 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786172985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:09:45.556520 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7721
I20260808 07:09:45.557890 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.548864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58623 (local address 127.18.63.126:43733)
0808 07:09:45.548932 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.548934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.549144 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:09:45.549785 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.549786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.549800 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.549847 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.550822 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.551315 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.552327 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.552529 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.552605 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.554825 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.554826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.554827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.554834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.556246 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.556792 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.556793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.556794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.556863 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.557191 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.557193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.557193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.557316 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.557425 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.557714 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.557800 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:09:45.558116 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.548832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:58623)
0808 07:09:45.549019 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.549061 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.549670 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.549931 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.549935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.550519 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:09:45.550717 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.550722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.551408 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.551409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.552235 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.552238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.552613 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.552614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.552752 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.553221 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:09:45.553230 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:09:45.554736 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:09:45.556339 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.556348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.556351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.556689 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.556954 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:09:45.556955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:09:45.556956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:09:45.557100 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:09:45.557419 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:09:45.557427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:09:45.557566 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.557677 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.557843 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260808 07:09:45.558611 7896 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43733
I20260808 07:09:45.558724 7896 heartbeater.cc:464] Registering TS with master...
I20260808 07:09:45.558902 7896 heartbeater.cc:511] Master 127.18.63.126:43733 requested a full tablet report, sending...
I20260808 07:09:45.559321 7299 ts_manager.cc:194] Registered new tserver with Master: 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
I20260808 07:09:45.559684 7299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:58623
I20260808 07:09:45.563877 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.556943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.18.63.126:43733)
0808 07:09:45.557068 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.557070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.557189 (+ 119us) server_negotiation.cc:407] Connection header received
0808 07:09:45.557268 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.557269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.557281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.557371 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.558270 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.558954 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.560013 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.560204 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.560284 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.561546 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.561547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.561548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.561555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.562734 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.563083 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.563085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.563085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.563145 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.563362 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.563363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.563363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.563469 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.563653 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.563690 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.563773 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260808 07:09:45.564613 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:09:45.567077 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.564941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.18.63.126:43733)
0808 07:09:45.564988 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.564990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.565122 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:09:45.565170 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.565170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.565198 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.565257 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:45.565379 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.565829 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.566524 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.566711 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.566760 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.566886 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:45.566924 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 07:09:45.567037 (+ 113us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53524: token expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 07:09:45.567144 7899 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53524: token expired
W20260808 07:09:45.567997 7204 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.63.126:43733)
I20260808 07:09:45.573189 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.568223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.18.63.126:43733)
0808 07:09:45.568273 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.568274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.568289 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:09:45.568328 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.568328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.568335 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.568381 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:09:45.568807 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.569219 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.569929 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.570094 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.570140 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.571124 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:09:45.571125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:09:45.571125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:09:45.571131 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:09:45.572278 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.572609 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.572610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.572610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.572659 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:09:45.572816 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:09:45.572816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:09:45.572816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:09:45.572903 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:09:45.572973 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.573054 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.573109 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 07:09:45.574352 7299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:09:45.576282 7299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:09:45.579990 7458 tablet_service.cc:1467] Processing CreateTablet for tablet ba7021d6bca6488fb835781bc516299e (DEFAULT_TABLE table=test-table [id=054f423c57bd44c9a9f6e586b886e19d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:45.580235 7458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7021d6bca6488fb835781bc516299e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:45.581789 7407 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Bootstrap starting.
I20260808 07:09:45.582129 7407 tablet_bootstrap.cc:654] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:45.582921 7407 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: No bootstrap required, opened a new log
I20260808 07:09:45.582993 7407 ts_tablet_manager.cc:1397] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:09:45.583372 7407 raft_consensus.cc:348] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.583480 7407 raft_consensus.cc:374] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:45.583523 7407 raft_consensus.cc:729] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e855d76a032f4c19b4c065715b4821b5, State: Initialized, Role: FOLLOWER
I20260808 07:09:45.583573 7407 consensus_queue.cc:260] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.583657 7407 ts_tablet_manager.cc:1428] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 07:09:45.584306 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.579645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:32857 (local address 127.0.0.1:38708)
0808 07:09:45.579678 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.579689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.579730 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.580449 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.580451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.580453 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:45.580503 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.580504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.581851 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.581852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.583084 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.583087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.584027 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.584029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.584126 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.584143 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.584178 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.584227 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":5}
I20260808 07:09:45.584451 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.579751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33999 (local address 127.0.0.1:59654)
0808 07:09:45.580015 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.580026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.580077 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.580588 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.580598 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.580600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:45.580648 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.580649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.581996 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.581997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.583085 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.583087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.584178 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.584179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.584251 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.584255 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.584306 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.584351 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260808 07:09:45.584685 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.579982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.18.63.67:33999)
0808 07:09:45.580275 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.580336 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.580349 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:09:45.580391 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.580393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.580408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.580482 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:45.580735 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.581915 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.583180 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.583984 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.584054 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.584121 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.584573 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.584619 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
I20260808 07:09:45.584684 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.579812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.18.63.66:32857)
0808 07:09:45.580244 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.580249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.580262 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:09:45.580309 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.580311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.580327 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.580425 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:45.580580 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.581748 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.583276 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.583925 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.584446 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.584520 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.584554 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.584604 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":101,"threads_started":1}
I20260808 07:09:45.585294 7797 tablet_service.cc:1467] Processing CreateTablet for tablet ba7021d6bca6488fb835781bc516299e (DEFAULT_TABLE table=test-table [id=054f423c57bd44c9a9f6e586b886e19d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:45.585479 7635 tablet_service.cc:1467] Processing CreateTablet for tablet ba7021d6bca6488fb835781bc516299e (DEFAULT_TABLE table=test-table [id=054f423c57bd44c9a9f6e586b886e19d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:09:45.585740 7797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7021d6bca6488fb835781bc516299e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:45.585843 7635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7021d6bca6488fb835781bc516299e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:09:45.588590 7908 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Bootstrap starting.
I20260808 07:09:45.588856 7909 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Bootstrap starting.
I20260808 07:09:45.589362 7908 tablet_bootstrap.cc:654] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:45.589636 7909 tablet_bootstrap.cc:654] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Neither blocks nor log segments found. Creating new log.
I20260808 07:09:45.590034 7908 log.cc:824] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:45.590282 7909 log.cc:824] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:09:45.591101 7908 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: No bootstrap required, opened a new log
I20260808 07:09:45.591176 7908 ts_tablet_manager.cc:1397] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:09:45.591228 7909 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: No bootstrap required, opened a new log
I20260808 07:09:45.591316 7909 ts_tablet_manager.cc:1397] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:09:45.592648 7908 raft_consensus.cc:348] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.592821 7908 raft_consensus.cc:374] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:45.592864 7908 raft_consensus.cc:729] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f9257212bf458a90bca526a5c9f5c2, State: Initialized, Role: FOLLOWER
I20260808 07:09:45.592983 7908 consensus_queue.cc:260] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.593290 7908 ts_tablet_manager.cc:1428] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:09:45.593319 7896 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
I20260808 07:09:45.593227 7909 raft_consensus.cc:348] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.593474 7909 raft_consensus.cc:374] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:09:45.593521 7909 raft_consensus.cc:729] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57209da1fc4f4797b11f20b50282bdcd, State: Initialized, Role: FOLLOWER
I20260808 07:09:45.593629 7909 consensus_queue.cc:260] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.593902 7909 ts_tablet_manager.cc:1428] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 07:09:45.593914 7718 heartbeater.cc:503] Master 127.18.63.126:43733 was elected leader, sending a full tablet report...
I20260808 07:09:45.626964 7914 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:09:45.627069 7914 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.627398 7914 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999), e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595)
I20260808 07:09:45.630609 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.627611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:54093 (local address 127.18.63.67:33999)
0808 07:09:45.627662 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.627664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.627676 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:09:45.627745 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.627746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.627756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.627824 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:45.628042 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.628628 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.629715 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.630239 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.630295 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.630302 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.630491 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.630523 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:09:45.630615 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.627555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33999 (local address 127.18.63.66:54093)
0808 07:09:45.627621 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.627634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.627686 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.627848 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.627849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.627851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:45.627931 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.627933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.628704 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.628705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.629630 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.629633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.630312 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.630313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.630400 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.630404 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.630467 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.630519 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 07:09:45.630936 7829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7021d6bca6488fb835781bc516299e" candidate_uuid: "57209da1fc4f4797b11f20b50282bdcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f9257212bf458a90bca526a5c9f5c2" is_pre_election: true
I20260808 07:09:45.631067 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.627559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:45595 (local address 127.18.63.66:43649)
0808 07:09:45.627775 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.627786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.627858 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.628076 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:09:45.628077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:09:45.628078 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:09:45.628178 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.628180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.629052 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.629053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.630001 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:09:45.630004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.630823 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.630824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:09:45.630891 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.630894 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:09:45.630941 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:09:45.630988 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":45,"thread_start_us":118,"threads_started":1}
I20260808 07:09:45.631063 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.627656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:43649 (local address 127.18.63.65:45595)
0808 07:09:45.627699 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.627701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.627833 (+ 132us) server_negotiation.cc:407] Connection header received
0808 07:09:45.627926 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.627928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.627957 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.628027 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:09:45.628274 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.628972 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.630094 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.630736 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.630807 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.630893 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.630958 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.631000 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:09:45.631124 7829 raft_consensus.cc:2454] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57209da1fc4f4797b11f20b50282bdcd in term 0.
I20260808 07:09:45.631379 7588 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd; no voters:
I20260808 07:09:45.631371 7488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7021d6bca6488fb835781bc516299e" candidate_uuid: "57209da1fc4f4797b11f20b50282bdcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e855d76a032f4c19b4c065715b4821b5" is_pre_election: true
I20260808 07:09:45.631520 7488 raft_consensus.cc:2454] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57209da1fc4f4797b11f20b50282bdcd in term 0.
I20260808 07:09:45.631529 7914 raft_consensus.cc:2781] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:09:45.631584 7914 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:09:45.631623 7914 raft_consensus.cc:3037] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:45.632364 7914 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.632517 7914 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [CANDIDATE]: Term 1 election: Requested vote from peers 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999), e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595)
I20260808 07:09:45.632687 7829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7021d6bca6488fb835781bc516299e" candidate_uuid: "57209da1fc4f4797b11f20b50282bdcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f9257212bf458a90bca526a5c9f5c2"
I20260808 07:09:45.632711 7488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7021d6bca6488fb835781bc516299e" candidate_uuid: "57209da1fc4f4797b11f20b50282bdcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e855d76a032f4c19b4c065715b4821b5"
I20260808 07:09:45.632763 7829 raft_consensus.cc:3037] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:45.632802 7488 raft_consensus.cc:3037] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:09:45.633550 7829 raft_consensus.cc:2454] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57209da1fc4f4797b11f20b50282bdcd in term 1.
I20260808 07:09:45.633725 7488 raft_consensus.cc:2454] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57209da1fc4f4797b11f20b50282bdcd in term 1.
I20260808 07:09:45.633731 7588 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd; no voters:
I20260808 07:09:45.633828 7914 raft_consensus.cc:2781] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:09:45.634016 7914 raft_consensus.cc:686] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 1 LEADER]: Becoming Leader. State: Replica: 57209da1fc4f4797b11f20b50282bdcd, State: Running, Role: LEADER
I20260808 07:09:45.634130 7914 consensus_queue.cc:237] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:09:45.635023 7298 catalog_manager.cc:5495] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd reported cstate change: term changed from 0 to 1, leader changed from <none> to 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66). New cstate: current_term: 1 leader_uuid: "57209da1fc4f4797b11f20b50282bdcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } health_report { overall_health: UNKNOWN } } }
I20260808 07:09:45.642858 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.640500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.18.63.66:32857)
0808 07:09:45.640550 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:09:45.640552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:09:45.640562 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:09:45.640608 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:09:45.640609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:09:45.640633 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:09:45.640706 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:09:45.640828 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.641296 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.642072 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:09:45.642266 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:09:45.642349 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:09:45.642510 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:09:45.642608 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:09:45.642690 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:09:45.642730 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:09:45.642755 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:09:45.644529 7488 raft_consensus.cc:1261] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Refusing update from remote peer 57209da1fc4f4797b11f20b50282bdcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:45.644537 7829 raft_consensus.cc:1261] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Refusing update from remote peer 57209da1fc4f4797b11f20b50282bdcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:09:45.644865 7914 consensus_queue.cc:1035] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:45.645020 7916 consensus_queue.cc:1035] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:09:45.647589 7922 mvcc.cc:205] Tried to move back new op lower bound from 7316164549197492224 to 7316164549158010880. Current Snapshot: MvccSnapshot[applied={T|T < 7316164549197492224}]
I20260808 07:09:45.649657 7299 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:45.649755 7299 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:09:45.650442 7299 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Sending DeleteTablet for 3 replicas of tablet ba7021d6bca6488fb835781bc516299e
I20260808 07:09:45.650780 7458 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7021d6bca6488fb835781bc516299e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:45 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:49144
I20260808 07:09:45.650787 7797 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7021d6bca6488fb835781bc516299e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:45 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:59654
I20260808 07:09:45.650863 7635 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7021d6bca6488fb835781bc516299e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:09:45 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38708
I20260808 07:09:45.650997 7931 tablet_replica.cc:330] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: stopping tablet replica
I20260808 07:09:45.651053 7561 tablet_replica.cc:330] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: stopping tablet replica
I20260808 07:09:45.651108 7931 raft_consensus.cc:2229] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:45.651122 7561 raft_consensus.cc:2229] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:09:45.651181 7561 raft_consensus.cc:2258] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:45.651219 7931 raft_consensus.cc:2258] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:45.651443 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 7346
I20260808 07:09:45.658946 7931 ts_tablet_manager.cc:1905] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:09:45.661007 7931 ts_tablet_manager.cc:1918] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:09:45.661095 7931 log.cc:1196] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/ba7021d6bca6488fb835781bc516299e
I20260808 07:09:45.661468 7931 ts_tablet_manager.cc:1939] T ba7021d6bca6488fb835781bc516299e P 20f9257212bf458a90bca526a5c9f5c2: Deleting consensus metadata
I20260808 07:09:45.661947 7280 catalog_manager.cc:4841] TS 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): tablet ba7021d6bca6488fb835781bc516299e (table test-table [id=054f423c57bd44c9a9f6e586b886e19d]) successfully deleted
I20260808 07:09:45.664565 7932 tablet_replica.cc:330] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: stopping tablet replica
I20260808 07:09:45.664706 7932 raft_consensus.cc:2229] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:09:45.664820 7932 raft_consensus.cc:2258] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:09:45.665246 7932 ts_tablet_manager.cc:1905] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 07:09:45.665701 7278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:45595 (error 108)
W20260808 07:09:45.666360 7904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:09:45.666067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:45595 (local address 127.0.0.1:49160)
0808 07:09:45.666125 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.666139 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:09:45.666185 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:09:45.666310 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:45595: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:09:45.666682 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 7553
W20260808 07:09:45.666610 7278 catalog_manager.cc:4584] TS e855d76a032f4c19b4c065715b4821b5 (127.18.63.65:45595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba7021d6bca6488fb835781bc516299e: Network error: Client connection negotiation failed: client connection to 127.18.63.65:45595: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:09:45.679589 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:09:45.679426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:32857 (local address 127.0.0.1:38736)
0808 07:09:45.679477 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:09:45.679525 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.66:32857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 07:09:45.679629 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 7721
I20260808 07:09:45.688076 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 7260
I20260808 07:10:05.697387 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--webserver_interface=127.18.63.126
--webserver_port=35581
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:05.784948 7935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:05.785195 7935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:05.785236 7935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:05.786432 7935 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 07:10:05.786509 7935 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 07:10:05.786545 7935 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 07:10:05.786567 7935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:05.786602 7935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:05.786635 7935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:05.786669 7935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:05.786701 7935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:05.788213 7935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=35581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:05.788522 7935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:05.788769 7935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:05.794181 7942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:05.794247 7935 server_base.cc:1034] running on GCE node
W20260808 07:10:05.794209 7944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:05.794260 7946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:05.794673 7935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:05.794955 7935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:05.796166 7935 hybrid_clock.cc:648] HybridClock initialized: now 1786173005796132 us; error 50 us; skew 500 ppm
Aug 08 07:10:05 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:05.798941 7935 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:05.799567 7935 webserver.cc:460] Webserver started at http://127.18.63.126:35581/ using document root <none> and password file <none>
I20260808 07:10:05.799804 7935 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:05.799875 7935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:05.801285 7935 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:05.802117 7956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:05.802347 7935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:05.802507 7935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "eace4f6291964bf9985ead84a85ad775"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "2fa59325b1726c0be859b4bcff1f4d3d"
server_key_iv: "d4a37b16efff26d139504dbb1ab3dedb"
server_key_version: "encryptionkey@0"
I20260808 07:10:05.802866 7935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:05.818871 7935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:05.821213 7935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:05.821440 7935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:05.825771 7935 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43733
I20260808 07:10:05.825824 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43733 every 8 connection(s)
I20260808 07:10:05.826216 7935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:05.827749 8028 sys_catalog.cc:263] Verifying existing consensus state
I20260808 07:10:05.828603 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap starting.
I20260808 07:10:05.833861 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 7935
I20260808 07:10:05.834194 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:45595
--local_ip_for_outbound_sockets=127.18.63.65
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=35877
--webserver_interface=127.18.63.65
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:10:05.838855 8028 log.cc:824] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:05.841061 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:10:05.841435 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Bootstrap complete.
I20260808 07:10:05.843719 8028 raft_consensus.cc:348] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:10:05.843943 8028 raft_consensus.cc:729] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Initialized, Role: FOLLOWER
I20260808 07:10:05.844105 8028 consensus_queue.cc:260] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:10:05.844192 8028 raft_consensus.cc:388] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:05.844244 8028 raft_consensus.cc:482] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:05.844297 8028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 2 FOLLOWER]: Advancing to term 3
I20260808 07:10:05.845257 8028 raft_consensus.cc:504] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:10:05.845391 8028 leader_election.cc:302] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eace4f6291964bf9985ead84a85ad775; no voters:
I20260808 07:10:05.845588 8028 leader_election.cc:288] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 07:10:05.845625 8033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 07:10:05.845744 8033 raft_consensus.cc:686] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [term 3 LEADER]: Becoming Leader. State: Replica: eace4f6291964bf9985ead84a85ad775, State: Running, Role: LEADER
I20260808 07:10:05.845830 8033 consensus_queue.cc:237] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } }
I20260808 07:10:05.845873 8028 sys_catalog.cc:564] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:05.846086 8034 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:10:05.846182 8034 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:05.846084 8033 sys_catalog.cc:455] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eace4f6291964bf9985ead84a85ad775. Latest consensus state: current_term: 3 leader_uuid: "eace4f6291964bf9985ead84a85ad775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eace4f6291964bf9985ead84a85ad775" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43733 } } }
I20260808 07:10:05.846395 8033 sys_catalog.cc:458] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:05.846664 8037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:05.847483 8037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:05.847641 8037 catalog_manager.cc:1259] Loaded cluster ID: d1f4b8ac00f94ce1916ec31d2e3e083e
I20260808 07:10:05.847875 8037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:05.849079 8037 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:05.849251 8037 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Loaded TSK: 0
I20260808 07:10:05.849300 8037 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eace4f6291964bf9985ead84a85ad775: Loaded TSK: 1
F20260808 07:10:05.849555 8037 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd09bcfc9ad google::LogMessage::Fail() at ??:0
@ 0x7fd09bcff88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd09bcfc4de google::LogMessage::Flush() at ??:0
@ 0x7fd09bcfda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd09e469104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd09e466d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd0a3c561cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd0a3c5648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd0a3c56594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fd0a3c1cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd0a3c26ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd09dbca0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd09dbbf954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd09ea236db start_thread at ??:0
@ 0x7fd09a7fa61f clone at ??:0
W20260808 07:10:05.940346 8031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:05.940544 8031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:05.940568 8031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:05.941795 8031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:05.941843 8031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:05.941870 8031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:05.943341 8031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:45595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=35877
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:05.943572 8031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:05.943819 8031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:05.949188 8077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:05.949210 8081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:05.949271 8079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:05.949479 8031 server_base.cc:1034] running on GCE node
I20260808 07:10:05.949766 8031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:05.950075 8031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:05.951260 8031 hybrid_clock.cc:648] HybridClock initialized: now 1786173005951246 us; error 29 us; skew 500 ppm
Aug 08 07:10:05 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:05.954432 8031 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:10:05.955022 8031 webserver.cc:460] Webserver started at http://127.18.63.65:35877/ using document root <none> and password file <none>
I20260808 07:10:05.955281 8031 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:05.955344 8031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:05.957183 8031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:05.958164 8089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:05.958431 8031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:05.958541 8031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "e855d76a032f4c19b4c065715b4821b5"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "77b1c86463233ec2e76870d7b8711033"
server_key_iv: "5d27b2359643b24ee6ec27c5b555ee10"
server_key_version: "encryptionkey@0"
I20260808 07:10:05.959012 8031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:05.973922 8031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:05.976444 8031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:05.976608 8031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:05.976909 8031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:05.977468 8101 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 07:10:05.978986 8031 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 07:10:05.979051 8031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 07:10:05.979084 8031 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 07:10:05.979998 8031 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 07:10:05.980049 8031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:05.980119 8101 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Bootstrap starting.
I20260808 07:10:05.987917 8031 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:45595
I20260808 07:10:05.988113 8239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:45595 every 8 connection(s)
I20260808 07:10:05.988406 8031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:05.990903 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8031
I20260808 07:10:05.991264 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:32857
--local_ip_for_outbound_sockets=127.18.63.66
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=36573
--webserver_interface=127.18.63.66
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:10:05.992189 8101 log.cc:824] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:05.995481 8101 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 07:10:05.995803 8101 tablet_bootstrap.cc:492] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Bootstrap complete.
I20260808 07:10:05.997205 8101 ts_tablet_manager.cc:1397] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.009s
I20260808 07:10:05.999073 8101 raft_consensus.cc:348] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:05.999485 8101 raft_consensus.cc:729] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e855d76a032f4c19b4c065715b4821b5, State: Initialized, Role: FOLLOWER
I20260808 07:10:05.999656 8101 consensus_queue.cc:260] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:05.999974 8101 ts_tablet_manager.cc:1428] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
W20260808 07:10:06.058358 8246 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:10:05.989139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:48271)
0808 07:10:05.989417 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:05.989470 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:05.990128 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:06.058143 (+ 68015us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
W20260808 07:10:06.058673 8242 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:43733 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:43733: Client connection negotiation failed: client connection to 127.18.63.126:43733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:10:06.059082 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.058948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:42781)
0808 07:10:06.059008 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.059057 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:10:06.059463 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.059364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:59151)
0808 07:10:06.059413 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.059442 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 07:10:06.059626 8242 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 07:10:06.080103 8249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:06.080296 8249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:06.080322 8249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:06.081617 8249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:06.081661 8249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:06.081688 8249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:06.083164 8249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=36573
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:06.083405 8249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:06.083639 8249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:06.089253 8263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:06.089354 8249 server_base.cc:1034] running on GCE node
W20260808 07:10:06.089279 8257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:06.089260 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:06.089699 8249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:06.089954 8249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:06.091099 8249 hybrid_clock.cc:648] HybridClock initialized: now 1786173006091073 us; error 44 us; skew 500 ppm
Aug 08 07:10:06 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:06.093657 8249 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:10:06.094099 8249 webserver.cc:460] Webserver started at http://127.18.63.66:36573/ using document root <none> and password file <none>
I20260808 07:10:06.094369 8249 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:06.094441 8249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:06.095899 8249 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:06.096621 8272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:06.096863 8249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:06.096978 8249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "57209da1fc4f4797b11f20b50282bdcd"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "8d51687d20c3b544943ef18fabae0297"
server_key_iv: "8dfd88a5f174e2b0db25d822969470a0"
server_key_version: "encryptionkey@0"
I20260808 07:10:06.097376 8249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:06.105595 8249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:06.107913 8249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:06.108096 8249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:06.108340 8249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:06.108803 8282 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 07:10:06.110409 8282 ts_tablet_manager.cc:1852] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 07:10:06.110689 8282 ts_tablet_manager.cc:1905] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:06.112848 8282 ts_tablet_manager.cc:1918] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:06.112928 8282 log.cc:1196] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/ba7021d6bca6488fb835781bc516299e
I20260808 07:10:06.113269 8282 ts_tablet_manager.cc:1939] T ba7021d6bca6488fb835781bc516299e P 57209da1fc4f4797b11f20b50282bdcd: Deleting consensus metadata
I20260808 07:10:06.113583 8249 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 07:10:06.113651 8249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 07:10:06.113695 8249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:06.113724 8249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:06.120113 8249 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:32857
I20260808 07:10:06.120223 8419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:32857 every 8 connection(s)
I20260808 07:10:06.120507 8249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:06.121660 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.121228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:36107)
0808 07:10:06.121445 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.121517 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
W20260808 07:10:06.121928 8421 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:43733 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:43733: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
I20260808 07:10:06.122215 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.122137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:33507)
0808 07:10:06.122170 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.122198 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 07:10:06.122560 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.122432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:43271)
0808 07:10:06.122521 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.122545 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 07:10:06.122678 8421 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:10:06.128831 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8249
I20260808 07:10:06.129128 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:33999
--local_ip_for_outbound_sockets=127.18.63.67
--tserver_master_addrs=127.18.63.126:43733
--webserver_port=43457
--webserver_interface=127.18.63.67
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:06.215246 8426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:06.215452 8426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:06.215483 8426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:06.216646 8426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:06.216688 8426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:06.216715 8426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:06.218261 8426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:33999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=43457
--tserver_master_addrs=127.18.63.126:43733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:06.218529 8426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:06.218767 8426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:06.223989 8437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:06.223948 8440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:06.223948 8436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:06.224107 8426 server_base.cc:1034] running on GCE node
I20260808 07:10:06.224399 8426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:06.224603 8426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:06.225757 8426 hybrid_clock.cc:648] HybridClock initialized: now 1786173006225732 us; error 40 us; skew 500 ppm
Aug 08 07:10:06 dist-test-slave-bw88 krb5kdc[6562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173006, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:06.228737 8426 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:10:06.229274 8426 webserver.cc:460] Webserver started at http://127.18.63.67:43457/ using document root <none> and password file <none>
I20260808 07:10:06.229573 8426 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:06.229656 8426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:06.231087 8426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:06.231941 8447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:06.232184 8426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:06.232275 8426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "20f9257212bf458a90bca526a5c9f5c2"
format_stamp: "Formatted at 2026-08-08 07:09:24 on dist-test-slave-bw88"
server_key: "e2b6db2389622e834131590217d029f8"
server_key_iv: "ca3cdf85e3d0ad3a91f4418278b16576"
server_key_version: "encryptionkey@0"
I20260808 07:10:06.232607 8426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:06.265839 8426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:06.267848 8426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:06.267966 8426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:06.268172 8426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:06.268522 8426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:06.268582 8426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:06.268607 8426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:06.268659 8426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:06.274721 8426 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:33999
I20260808 07:10:06.274816 8590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:33999 every 8 connection(s)
I20260808 07:10:06.275182 8426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:06.275782 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8426
I20260808 07:10:06.276580 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.276049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:59791)
0808 07:10:06.276280 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.276382 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260808 07:10:06.276989 8591 heartbeater.cc:644] Failed to heartbeat to 127.18.63.126:43733 (0 consecutive failures): Network error: Failed to ping master at 127.18.63.126:43733: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
I20260808 07:10:06.277273 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.277179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:37307)
0808 07:10:06.277229 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.277255 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:10:06.277554 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:06.277480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:36371)
0808 07:10:06.277519 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:06.277539 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:10:06.277670 8591 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 07:10:07.060300 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.060025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:37885)
0808 07:10:07.060189 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:07.060231 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260808 07:10:07.123231 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.123032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:44673)
0808 07:10:07.123154 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:07.123193 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260808 07:10:07.144879 8597 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:07.145072 8597 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:07.145494 8597 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
Aug 08 07:10:07 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.66@KRBTEST.COM
Aug 08 07:10:07 dist-test-slave-bw88 krb5kdc[6562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173005, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.67@KRBTEST.COM
I20260808 07:10:07.158243 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.145742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33999 (local address 127.18.63.65:41663)
0808 07:10:07.145928 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:07.145939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:07.145983 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:07.146841 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:07.146844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:07.147289 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:07.147537 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:07.147544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.149227 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.149228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:07.150590 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:07.150594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.151464 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.151465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:07.151597 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:07.152256 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:07.152267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:07.154538 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:07.156603 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:07.156604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:07.156605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:07.156853 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:07.157175 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:07.157176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:07.157176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:07.157258 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:07.157763 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:07.157764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:07.157822 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 07:10:07.157933 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 07:10:07.158144 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":73,"thread_start_us":62,"threads_started":1}
I20260808 07:10:07.158243 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.145920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:41663 (local address 127.18.63.67:33999)
0808 07:10:07.146054 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:07.146068 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:07.146087 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:10:07.146564 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:07.146568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:07.146634 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:07.146769 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:07.147731 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.149104 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.150710 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.151320 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.151440 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:07.154659 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:07.154664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:07.154664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:07.154677 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:07.156484 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:07.156941 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:07.156944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:07.156944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:07.157092 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:07.157376 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:07.157377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:07.157377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:07.157676 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:07.157773 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:07.157996 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 07:10:07.158136 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260808 07:10:07.158589 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.145675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:32857 (local address 127.18.63.65:42217)
0808 07:10:07.145742 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:07.145757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:07.145829 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:07.146840 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:07.146843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:07.147287 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:07.147538 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:07.147545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.149229 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.149229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:07.150590 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:07.150594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.151430 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.151432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:07.151597 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:07.152283 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:07.152287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:07.154143 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:07.156381 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:07.156389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:07.156391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:07.156747 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:07.157037 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:07.157038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:07.157039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:07.157177 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:07.157623 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:07.157631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:07.157808 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 07:10:07.157952 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 07:10:07.158247 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":204}
I20260808 07:10:07.159072 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.145768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:42217 (local address 127.18.63.66:32857)
0808 07:10:07.145962 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:07.145975 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:07.146005 (+ 30us) server_negotiation.cc:407] Connection header received
0808 07:10:07.146582 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:07.146585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:07.146631 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:07.146802 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:07.147674 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.149096 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.150685 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:07.151320 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:07.151440 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:07.154262 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:07.154267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:07.154267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:07.154279 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:07.156285 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:07.156832 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:07.156836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:07.156837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:07.156963 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:07.157252 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:07.157253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:07.157254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:07.157541 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:07.157612 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:07.158760 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0808 07:10:07.158933 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
W20260808 07:10:07.159159 8095 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
W20260808 07:10:07.159734 8096 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
I20260808 07:10:07.159801 8096 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e855d76a032f4c19b4c065715b4821b5; no voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd
I20260808 07:10:07.159969 8597 raft_consensus.cc:2726] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:07.278249 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:07.278015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:59771)
0808 07:10:07.278151 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:07.278197 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260808 07:10:08.061201 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:08.060925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:39555)
0808 07:10:08.061096 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:08.061140 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260808 07:10:08.123975 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:08.123692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:60675)
0808 07:10:08.123897 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:08.123936 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":70,"threads_started":1}
I20260808 07:10:08.278947 8609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:08.278743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:42807)
0808 07:10:08.278869 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:08.278906 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260808 07:10:08.988312 8610 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:08.988407 8610 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:08.988655 8610 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
W20260808 07:10:08.989188 8096 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
W20260808 07:10:08.989257 8095 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
I20260808 07:10:08.989287 8096 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e855d76a032f4c19b4c065715b4821b5; no voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd
I20260808 07:10:08.989394 8610 raft_consensus.cc:2726] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:09.062016 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:09.061773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:47257)
0808 07:10:09.061926 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:09.061964 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260808 07:10:09.124677 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:09.124460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:34617)
0808 07:10:09.124599 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:09.124635 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260808 07:10:09.279675 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:09.279455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:43447)
0808 07:10:09.279583 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:09.279619 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260808 07:10:10.062862 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:10.062586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:45061)
0808 07:10:10.062774 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:10.062815 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260808 07:10:10.125500 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:10.125170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:38183)
0808 07:10:10.125394 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:10.125446 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260808 07:10:10.280349 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:10.280140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:41653)
0808 07:10:10.280272 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:10.280309 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 07:10:11.063725 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.063449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:34051)
0808 07:10:11.063625 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.063669 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260808 07:10:11.126317 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.126065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:41561)
0808 07:10:11.126218 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.126265 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260808 07:10:11.191042 8621 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:11.191159 8621 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:11.191413 8621 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
W20260808 07:10:11.191938 8096 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
W20260808 07:10:11.192062 8095 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
I20260808 07:10:11.192162 8095 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e855d76a032f4c19b4c065715b4821b5; no voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd
I20260808 07:10:11.192262 8621 raft_consensus.cc:2726] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:11.281179 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:11.280931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:46563)
0808 07:10:11.281079 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:11.281122 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
I20260808 07:10:12.064388 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:12.064178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:46057)
0808 07:10:12.064304 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:12.064339 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260808 07:10:12.127112 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:12.126884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:51705)
0808 07:10:12.127017 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:12.127064 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260808 07:10:12.281962 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:12.281689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:38755)
0808 07:10:12.281855 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:12.281898 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260808 07:10:13.065192 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.064952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:33573)
0808 07:10:13.065100 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.065142 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260808 07:10:13.127905 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.127679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:57459)
0808 07:10:13.127826 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.127865 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260808 07:10:13.282600 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:13.282401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:53511)
0808 07:10:13.282526 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:13.282558 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260808 07:10:13.998430 8631 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:13.998525 8631 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:13.998750 8631 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
W20260808 07:10:13.999207 8096 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
W20260808 07:10:13.999351 8095 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
I20260808 07:10:13.999414 8095 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e855d76a032f4c19b4c065715b4821b5; no voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd
I20260808 07:10:13.999509 8631 raft_consensus.cc:2726] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:14.065924 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:14.065708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:54459)
0808 07:10:14.065838 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:14.065875 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260808 07:10:14.128604 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:14.128389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:46933)
0808 07:10:14.128504 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:14.128561 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260808 07:10:14.283288 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:14.283064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:36121)
0808 07:10:14.283191 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:14.283228 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 07:10:15.066637 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.066382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:58577)
0808 07:10:15.066543 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.066587 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260808 07:10:15.129235 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.129014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:58047)
0808 07:10:15.129152 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.129189 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260808 07:10:15.283951 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:15.283734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:52329)
0808 07:10:15.283868 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:15.283903 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260808 07:10:16.028831 8641 raft_consensus.cc:482] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:16.028966 8641 raft_consensus.cc:504] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57209da1fc4f4797b11f20b50282bdcd" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 32857 } } peers { permanent_uuid: "20f9257212bf458a90bca526a5c9f5c2" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33999 } } peers { permanent_uuid: "e855d76a032f4c19b4c065715b4821b5" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 45595 } }
I20260808 07:10:16.029147 8641 leader_election.cc:288] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857), 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999)
W20260808 07:10:16.029606 8095 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20f9257212bf458a90bca526a5c9f5c2 (127.18.63.67:33999): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
W20260808 07:10:16.029700 8096 leader_election.cc:341] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57209da1fc4f4797b11f20b50282bdcd (127.18.63.66:32857): Not found: Tablet not found: ba7021d6bca6488fb835781bc516299e
I20260808 07:10:16.029805 8096 leader_election.cc:302] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e855d76a032f4c19b4c065715b4821b5; no voters: 20f9257212bf458a90bca526a5c9f5c2, 57209da1fc4f4797b11f20b50282bdcd
I20260808 07:10:16.029894 8641 raft_consensus.cc:2726] T ba7021d6bca6488fb835781bc516299e P e855d76a032f4c19b4c065715b4821b5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 07:10:16.067461 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.067197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:43413)
0808 07:10:16.067368 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.067411 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260808 07:10:16.129938 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.129714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:51127)
0808 07:10:16.129858 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.129897 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260808 07:10:16.284627 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:16.284405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:41005)
0808 07:10:16.284550 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:16.284585 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260808 07:10:17.068168 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:17.067914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:59501)
0808 07:10:17.068083 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:17.068120 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260808 07:10:17.130716 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:17.130468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:51845)
0808 07:10:17.130618 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:17.130666 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260808 07:10:17.285446 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:17.285197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:50051)
0808 07:10:17.285360 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:17.285397 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260808 07:10:18.068955 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.068705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:39545)
0808 07:10:18.068860 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.068906 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260808 07:10:18.131443 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.131230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:47929)
0808 07:10:18.131361 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.131396 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260808 07:10:18.286204 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:18.285974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:33255)
0808 07:10:18.286112 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:18.286156 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 07:10:19.069808 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.069521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:43435)
0808 07:10:19.069712 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:19.069755 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":106,"threads_started":1}
I20260808 07:10:19.132300 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.132069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:38415)
0808 07:10:19.132212 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:19.132253 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 07:10:19.286988 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:19.286760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:37101)
0808 07:10:19.286905 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:19.286938 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260808 07:10:20.070588 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.070333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:46389)
0808 07:10:20.070490 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:20.070534 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260808 07:10:20.133162 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.132910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:40091)
0808 07:10:20.133070 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:20.133111 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260808 07:10:20.287665 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:20.287408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:39659)
0808 07:10:20.287569 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:20.287618 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260808 07:10:21.071412 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:21.071178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.65:36111)
0808 07:10:21.071327 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:21.071364 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260808 07:10:21.134042 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:21.133788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.66:57275)
0808 07:10:21.133949 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:21.133989 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
I20260808 07:10:21.278788 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8031
I20260808 07:10:21.288471 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:21.288232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43733 (local address 127.18.63.67:39539)
0808 07:10:21.288381 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:21.288424 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.126:43733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260808 07:10:21.291867 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8249
I20260808 07:10:21.303624 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8426
2026-08-08T07:10:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58538 ms)
[----------] 30 tests from SecurityITest (242003 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:21 dist-test-slave-bw88 krb5kdc[8665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:21 dist-test-slave-bw88 krb5kdc[8665](info): set up 2 sockets
Aug 08 07:10:21 dist-test-slave-bw88 krb5kdc[8665](info): commencing operation
krb5kdc: starting...
W20260808 07:10:22.499419 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:22 dist-test-slave-bw88 krb5kdc[8665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:22.626534 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43033
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:43033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:22.710421 8681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:22.710680 8681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:22.710718 8681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:22.711867 8681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:22.711941 8681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:22.711964 8681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:22.711985 8681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:22.712010 8681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:22.713552 8681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:43033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:22.713837 8681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:22.714077 8681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:22.719481 8688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:22.719522 8691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:22.719584 8689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:22.719632 8681 server_base.cc:1034] running on GCE node
I20260808 07:10:22.719996 8681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:22.720254 8681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:22.721408 8681 hybrid_clock.cc:648] HybridClock initialized: now 1786173022721384 us; error 50 us; skew 500 ppm
Aug 08 07:10:22 dist-test-slave-bw88 krb5kdc[8665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173022, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:22.723907 8681 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:22.724385 8681 webserver.cc:460] Webserver started at http://127.18.63.126:44801/ using document root <none> and password file <none>
I20260808 07:10:22.724645 8681 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:22.724715 8681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:22.724865 8681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:22.726074 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "d49297c08e054d678e1bd66abbad9909"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "2f2d8475568ecd20b744f4b39b7a2514"
server_key_iv: "24e9b1ee650be322cb95a9640a754eb7"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.726451 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "d49297c08e054d678e1bd66abbad9909"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "2f2d8475568ecd20b744f4b39b7a2514"
server_key_iv: "24e9b1ee650be322cb95a9640a754eb7"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.727988 8681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:22.729413 8699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:22.729645 8681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:22.729734 8681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "d49297c08e054d678e1bd66abbad9909"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "2f2d8475568ecd20b744f4b39b7a2514"
server_key_iv: "24e9b1ee650be322cb95a9640a754eb7"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.729804 8681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:22.756600 8681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:22.758718 8681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:22.758881 8681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:22.763417 8681 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:43033
I20260808 07:10:22.763517 8765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:43033 every 8 connection(s)
I20260808 07:10:22.763856 8681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:22.764806 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:22.767292 8766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: Bootstrap starting.
I20260808 07:10:22.767910 8766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:22.768227 8766 log.cc:824] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:22.769164 8766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: No bootstrap required, opened a new log
I20260808 07:10:22.770892 8766 raft_consensus.cc:348] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } }
I20260808 07:10:22.771036 8766 raft_consensus.cc:374] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:22.771060 8766 raft_consensus.cc:729] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d49297c08e054d678e1bd66abbad9909, State: Initialized, Role: FOLLOWER
I20260808 07:10:22.771203 8766 consensus_queue.cc:260] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } }
I20260808 07:10:22.771291 8766 raft_consensus.cc:388] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:22.771314 8766 raft_consensus.cc:482] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:22.771389 8766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:22.771740 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8681
I20260808 07:10:22.771834 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:22.772068 18685 external_mini_cluster.cc:1428] Setting key 0507ae5f7ca4e70a9d6ede99b1500f3e
I20260808 07:10:22.772053 8766 raft_consensus.cc:504] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } }
I20260808 07:10:22.772176 8766 leader_election.cc:302] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d49297c08e054d678e1bd66abbad9909; no voters:
I20260808 07:10:22.772390 8766 leader_election.cc:288] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:22.772415 8769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:22.772663 8769 raft_consensus.cc:686] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [term 1 LEADER]: Becoming Leader. State: Replica: d49297c08e054d678e1bd66abbad9909, State: Running, Role: LEADER
I20260808 07:10:22.772683 8766 sys_catalog.cc:564] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:22.772760 8769 consensus_queue.cc:237] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } }
I20260808 07:10:22.773365 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d49297c08e054d678e1bd66abbad9909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } } }
I20260808 07:10:22.773464 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:22.773608 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d49297c08e054d678e1bd66abbad9909. Latest consensus state: current_term: 1 leader_uuid: "d49297c08e054d678e1bd66abbad9909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d49297c08e054d678e1bd66abbad9909" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 43033 } } }
I20260808 07:10:22.773664 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:22.773751 8782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:22.774343 8782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:22.777038 8782 catalog_manager.cc:1347] Generated new cluster ID: 802af5675d22482d88d8c8fb8c280a65
I20260808 07:10:22.777096 8782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:22 dist-test-slave-bw88 krb5kdc[8665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:22.784152 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:22.772552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.18.63.126:43033)
0808 07:10:22.772829 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:22.772855 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:22.773057 (+ 202us) server_negotiation.cc:407] Connection header received
0808 07:10:22.773230 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:22.773238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:22.773299 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:22.774250 (+ 951us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:22.774395 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.775860 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.776831 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.777501 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.777622 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:22.780432 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:22.780436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:22.780436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:22.780448 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:22.782431 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:22.782861 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:22.782864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:22.782865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:22.782986 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:22.783234 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:22.783235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:22.783235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:22.783521 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:22.783683 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:22.783800 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:10:22.783931 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
I20260808 07:10:22.785061 8782 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:22.785583 8782 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:22.789772 8782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: Generated new TSK 0
I20260808 07:10:22.789952 8782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:10:22.818331 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:22.903477 8800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:22.903692 8800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:22.903715 8800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:22.904883 8800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:22.904927 8800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:22.904953 8800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:22.906487 8800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:22.906765 8800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:22.907028 8800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:22.913025 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:22.913043 8809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:22.913100 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:22.913393 8800 server_base.cc:1034] running on GCE node
I20260808 07:10:22.913676 8800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:22.913945 8800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:22.915144 8800 hybrid_clock.cc:648] HybridClock initialized: now 1786173022915129 us; error 26 us; skew 500 ppm
Aug 08 07:10:22 dist-test-slave-bw88 krb5kdc[8665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173022, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:22.918128 8800 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:10:22.918551 8800 webserver.cc:460] Webserver started at http://127.18.63.65:42953/ using document root <none> and password file <none>
I20260808 07:10:22.918771 8800 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:22.918814 8800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:22.918912 8800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:22.919986 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "672d8fa7357e4e9384f0115f47299152"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "538eb9a1f1419f9ca52862f1f67a8e51"
server_key_iv: "357a78983782e7e2ae6e3df6b5664ac8"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.920320 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "672d8fa7357e4e9384f0115f47299152"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "538eb9a1f1419f9ca52862f1f67a8e51"
server_key_iv: "357a78983782e7e2ae6e3df6b5664ac8"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.921769 8800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:10:22.922581 8817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:22.922757 8800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:22.922844 8800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "672d8fa7357e4e9384f0115f47299152"
format_stamp: "Formatted at 2026-08-08 07:10:22 on dist-test-slave-bw88"
server_key: "538eb9a1f1419f9ca52862f1f67a8e51"
server_key_iv: "357a78983782e7e2ae6e3df6b5664ac8"
server_key_version: "encryptionkey@0"
I20260808 07:10:22.922899 8800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:22.935606 8800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:22.937928 8800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:22.938040 8800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:22.938220 8800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:22.938562 8800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:22.938598 8800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:22.938621 8800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:22.938668 8800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:22.944917 8800 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:46025
I20260808 07:10:22.945016 8956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:46025 every 8 connection(s)
I20260808 07:10:22.945343 8800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:22 dist-test-slave-bw88 krb5kdc[8665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173022, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:22.954113 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8800
I20260808 07:10:22.954269 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:22.954532 18685 external_mini_cluster.cc:1428] Setting key 79a4938bdb6bb5b68f0248dbdc50a47b
I20260808 07:10:22.954923 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:22.946192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:44269 (local address 127.18.63.126:43033)
0808 07:10:22.946265 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:22.946267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:22.946474 (+ 207us) server_negotiation.cc:407] Connection header received
0808 07:10:22.946981 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:22.946982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:22.946992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:22.947049 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:22.948022 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.948531 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.949766 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.949958 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.950015 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:22.952085 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:22.952086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:22.952086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:22.952093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:22.953326 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:22.953785 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:22.953786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:22.953786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:22.953838 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:22.954168 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:22.954169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:22.954169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:22.954338 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:22.954406 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:22.954741 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 07:10:22.954830 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:10:22.955107 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:22.946141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43033 (local address 127.18.63.65:44269)
0808 07:10:22.946335 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:22.946377 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:22.946884 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:22.947106 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:22.947110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:22.947714 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:22.947908 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:22.947922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.948633 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.948634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:22.949678 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:22.949681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:22.950057 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:22.950059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:22.950168 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:22.950585 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:22.950593 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:22.951987 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:22.953472 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:22.953479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:22.953481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:22.953712 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:22.953929 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:22.953930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:22.953930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:22.954085 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:22.954399 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:22.954408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:22.954540 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 07:10:22.954717 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 07:10:22.954885 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260808 07:10:22.955664 8958 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43033
I20260808 07:10:22.955775 8958 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:22.955976 8958 heartbeater.cc:511] Master 127.18.63.126:43033 requested a full tablet report, sending...
I20260808 07:10:22.956576 8718 ts_manager.cc:194] Registered new tserver with Master: 672d8fa7357e4e9384f0115f47299152 (127.18.63.65:46025)
I20260808 07:10:22.957506 8718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:44269
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:10:22.982177 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:23.067199 8965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:23.067411 8965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:23.067435 8965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:23.068511 8965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:23.068547 8965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:23.068572 8965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:23.070055 8965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:23.070240 8965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:23.070415 8965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:23.075810 8972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.075771 8974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.075767 8971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.075951 8965 server_base.cc:1034] running on GCE node
I20260808 07:10:23.076191 8965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:23.076437 8965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:23.077592 8965 hybrid_clock.cc:648] HybridClock initialized: now 1786173023077560 us; error 54 us; skew 500 ppm
Aug 08 07:10:23 dist-test-slave-bw88 krb5kdc[8665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:23.080117 8965 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:10:23.080600 8965 webserver.cc:460] Webserver started at http://127.18.63.66:39507/ using document root <none> and password file <none>
I20260808 07:10:23.080832 8965 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:23.080875 8965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:23.080964 8965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:23.082098 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "6b09e2c43dc24202bce96a6f732f9a18"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "6db10f7d1bdb765a680bcddbaeb0ab3e"
server_key_iv: "da70d545b7d376cc817243bd2491dc41"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.082414 8965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "6b09e2c43dc24202bce96a6f732f9a18"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "6db10f7d1bdb765a680bcddbaeb0ab3e"
server_key_iv: "da70d545b7d376cc817243bd2491dc41"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.083736 8965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:23.084471 8986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.084692 8965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:23.084800 8965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "6b09e2c43dc24202bce96a6f732f9a18"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "6db10f7d1bdb765a680bcddbaeb0ab3e"
server_key_iv: "da70d545b7d376cc817243bd2491dc41"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.084882 8965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:23.097021 8965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:23.099048 8965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:23.099208 8965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:23.099474 8965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:23.099813 8965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:23.099870 8965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.099912 8965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:23.099941 8965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.106274 8965 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:43877
I20260808 07:10:23.106328 9126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:43877 every 8 connection(s)
I20260808 07:10:23.106642 8965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:23.107532 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 8965
I20260808 07:10:23.107651 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:23.107882 18685 external_mini_cluster.cc:1428] Setting key 479b255731f15c704221e7f1849a8114
Aug 08 07:10:23 dist-test-slave-bw88 krb5kdc[8665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:23.117251 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.107450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:37059 (local address 127.18.63.126:43033)
0808 07:10:23.107524 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.107527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.107755 (+ 228us) server_negotiation.cc:407] Connection header received
0808 07:10:23.108419 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.108420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.108431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.108502 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.109667 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.110403 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.111874 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.112068 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.112129 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.114358 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.114360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.114360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:23.114368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.115845 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.116270 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.116271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.116272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.116333 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.116594 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.116595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.116595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.116714 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.116787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.117068 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.117159 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 07:10:23.117481 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.107354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43033 (local address 127.18.63.66:37059)
0808 07:10:23.107636 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.107679 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.108310 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.108636 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.108642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.109299 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:23.109573 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.109579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.110502 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.110502 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.111780 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.111784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.112152 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.112153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.112285 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.112825 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:23.112834 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:23.114255 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:23.115946 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:23.115954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:23.115956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:23.116189 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:23.116403 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:23.116404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:23.116404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:23.116518 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:23.116790 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:23.116798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:23.116937 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.117047 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.117215 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
I20260808 07:10:23.117992 9127 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43033
I20260808 07:10:23.118091 9127 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:23.118302 9127 heartbeater.cc:511] Master 127.18.63.126:43033 requested a full tablet report, sending...
I20260808 07:10:23.118705 8718 ts_manager.cc:194] Registered new tserver with Master: 6b09e2c43dc24202bce96a6f732f9a18 (127.18.63.66:43877)
I20260808 07:10:23.119184 8718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:37059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:10:23.135943 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:23.221434 9137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:23.221644 9137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:23.221668 9137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:23.222831 9137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:23.222877 9137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:23.222903 9137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:23.224466 9137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:43033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:23.224738 9137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:23.225029 9137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:23.230479 9144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.230569 9145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.230734 9137 server_base.cc:1034] running on GCE node
W20260808 07:10:23.230491 9148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.231016 9137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:23.231308 9137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:23.232473 9137 hybrid_clock.cc:648] HybridClock initialized: now 1786173023232465 us; error 43 us; skew 500 ppm
Aug 08 07:10:23 dist-test-slave-bw88 krb5kdc[8665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:23.235136 9137 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:10:23.235579 9137 webserver.cc:460] Webserver started at http://127.18.63.67:33247/ using document root <none> and password file <none>
I20260808 07:10:23.235833 9137 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:23.235903 9137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:23.236076 9137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:23.237161 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "bb52d56ad64547f3bd5c1f92efab3e36"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "0c2b842d4c7bf70874190b1a28fac73a"
server_key_iv: "13a2237df5dea7cd301f93be71d9cc6f"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.237609 9137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "bb52d56ad64547f3bd5c1f92efab3e36"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "0c2b842d4c7bf70874190b1a28fac73a"
server_key_iv: "13a2237df5dea7cd301f93be71d9cc6f"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.239070 9137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:23.239899 9158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.240116 9137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:23.240226 9137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "bb52d56ad64547f3bd5c1f92efab3e36"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "0c2b842d4c7bf70874190b1a28fac73a"
server_key_iv: "13a2237df5dea7cd301f93be71d9cc6f"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.240309 9137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:23.256700 9137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:23.258922 9137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:23.259114 9137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:23.259382 9137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:23.259743 9137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:23.259811 9137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.259845 9137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:23.259871 9137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.266219 9137 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:43549
I20260808 07:10:23.266325 9295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:43549 every 8 connection(s)
I20260808 07:10:23.266636 9137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:23.271952 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 9137
I20260808 07:10:23.272104 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:23.272344 18685 external_mini_cluster.cc:1428] Setting key 2601ae076651dd225e33213002d0ed10
Aug 08 07:10:23 dist-test-slave-bw88 krb5kdc[8665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:23.276574 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.267466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:40349 (local address 127.18.63.126:43033)
0808 07:10:23.267573 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.267575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.267749 (+ 174us) server_negotiation.cc:407] Connection header received
0808 07:10:23.268295 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.268296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.268307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.268361 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.269457 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.269950 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.270993 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.271180 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.271237 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.273991 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.273992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.273992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:23.273999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.275398 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.275773 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.275774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.275774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.275826 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.276050 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.276051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.276051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.276161 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.276212 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.276432 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.276489 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 07:10:23.276806 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.267402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:43033 (local address 127.18.63.67:40349)
0808 07:10:23.267614 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.267653 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.268203 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.268421 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.268424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.269089 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:23.269304 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.269310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.270056 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.270058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.270909 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.270911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.271261 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.271261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.271365 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.271961 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:23.271972 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:23.273871 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:23.275469 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:23.275476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:23.275477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:23.275703 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:23.275884 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:23.275885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:23.275885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:23.275994 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:23.276216 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:23.276221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:23.276331 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.276411 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.276566 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":126,"threads_started":1}
I20260808 07:10:23.277252 9296 heartbeater.cc:347] Connected to a master server at 127.18.63.126:43033
I20260808 07:10:23.277387 9296 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:23.277554 9296 heartbeater.cc:511] Master 127.18.63.126:43033 requested a full tablet report, sending...
I20260808 07:10:23.277899 8718 ts_manager.cc:194] Registered new tserver with Master: bb52d56ad64547f3bd5c1f92efab3e36 (127.18.63.67:43549)
I20260808 07:10:23.278218 8718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:40349
I20260808 07:10:23.284332 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:23.292282 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.285748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.18.63.126:43033)
0808 07:10:23.285807 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.285809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.285915 (+ 106us) server_negotiation.cc:407] Connection header received
0808 07:10:23.285970 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.285971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.285979 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.286028 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.286557 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.287012 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.287740 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.287930 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.288065 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.289323 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.289325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.289325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:23.289333 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.291047 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.291436 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.291437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.291438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.291508 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:23.291723 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:23.291724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:23.291725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:23.291876 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.291985 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.292101 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.292182 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:23.293901 8718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:23.295619 8718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:23.302851 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.18.63.67:43549)
0808 07:10:23.298855 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.298859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.298870 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:23.298916 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.298918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.298936 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.299026 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:23.299230 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300457 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.301814 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302342 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.302466 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.302535 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.302702 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.302755 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":90,"threads_started":1}
I20260808 07:10:23.302886 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:43549 (local address 127.0.0.1:57960)
0808 07:10:23.298357 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.298401 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.298498 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.299040 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.299044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.299047 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:23.299135 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.299138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300587 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.300588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.301719 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.301721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302468 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.302470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.302564 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.302595 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.302684 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.302779 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":68,"thread_start_us":111,"threads_started":1}
I20260808 07:10:23.302884 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:43877 (local address 127.0.0.1:57630)
0808 07:10:23.299004 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.299013 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.299060 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.299390 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.299390 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.299393 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:23.299443 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.299445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300348 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.300349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.301686 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.301689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302642 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.302643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.302695 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.302699 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.302753 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.302789 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260808 07:10:23.302884 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46025 (local address 127.0.0.1:55126)
0808 07:10:23.298821 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.298833 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.298906 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.299210 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.299212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.299214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:23.299256 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.299258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300590 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.300591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.301616 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.301619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302564 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.302565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.302627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.302632 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.302664 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.302770 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260808 07:10:23.303254 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.18.63.65:46025)
0808 07:10:23.298820 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.298826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.298892 (+ 66us) server_negotiation.cc:407] Connection header received
0808 07:10:23.298980 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.298982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.299020 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.299122 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:23.299337 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300456 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.301698 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302455 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.303018 (+ 563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.303113 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.303139 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.303194 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260808 07:10:23.303294 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.298641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.18.63.66:43877)
0808 07:10:23.298820 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.298826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.299116 (+ 290us) server_negotiation.cc:407] Connection header received
0808 07:10:23.299208 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.299210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.299226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.299303 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:23.299526 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.300232 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.301798 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.302471 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.302551 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.302611 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.303169 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.303220 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260808 07:10:23.304064 9043 tablet_service.cc:1467] Processing CreateTablet for tablet c64cafa44918431d9272de25dacf7b52 (DEFAULT_TABLE table=test-table [id=c3a0ce21059a4b7d96b0564a4b38b70e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:23.304067 8875 tablet_service.cc:1467] Processing CreateTablet for tablet c64cafa44918431d9272de25dacf7b52 (DEFAULT_TABLE table=test-table [id=c3a0ce21059a4b7d96b0564a4b38b70e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:23.304334 9216 tablet_service.cc:1467] Processing CreateTablet for tablet c64cafa44918431d9272de25dacf7b52 (DEFAULT_TABLE table=test-table [id=c3a0ce21059a4b7d96b0564a4b38b70e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:23.304463 8875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64cafa44918431d9272de25dacf7b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:23.304461 9043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64cafa44918431d9272de25dacf7b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:23.304634 9216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64cafa44918431d9272de25dacf7b52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:23.307371 9313 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Bootstrap starting.
I20260808 07:10:23.308096 9315 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Bootstrap starting.
I20260808 07:10:23.308332 9313 tablet_bootstrap.cc:654] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:23.308830 9315 tablet_bootstrap.cc:654] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:23.308974 9313 log.cc:824] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:23.309458 9314 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Bootstrap starting.
I20260808 07:10:23.309552 9315 log.cc:824] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:23.310081 9313 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: No bootstrap required, opened a new log
I20260808 07:10:23.310142 9313 ts_tablet_manager.cc:1397] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:10:23.310105 9314 tablet_bootstrap.cc:654] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:23.310642 9314 log.cc:824] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:23.311587 9315 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: No bootstrap required, opened a new log
I20260808 07:10:23.311571 9314 tablet_bootstrap.cc:492] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: No bootstrap required, opened a new log
I20260808 07:10:23.311661 9314 ts_tablet_manager.cc:1397] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:23.311756 9315 ts_tablet_manager.cc:1397] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 07:10:23.312183 9313 raft_consensus.cc:348] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.312361 9313 raft_consensus.cc:374] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:23.312405 9313 raft_consensus.cc:729] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b09e2c43dc24202bce96a6f732f9a18, State: Initialized, Role: FOLLOWER
I20260808 07:10:23.312549 9313 consensus_queue.cc:260] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.312861 9127 heartbeater.cc:503] Master 127.18.63.126:43033 was elected leader, sending a full tablet report...
I20260808 07:10:23.312866 9313 ts_tablet_manager.cc:1428] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:23.313884 9314 raft_consensus.cc:348] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.313938 9315 raft_consensus.cc:348] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.314085 9314 raft_consensus.cc:374] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:23.314095 9315 raft_consensus.cc:374] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:23.314133 9315 raft_consensus.cc:729] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb52d56ad64547f3bd5c1f92efab3e36, State: Initialized, Role: FOLLOWER
I20260808 07:10:23.314129 9314 raft_consensus.cc:729] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672d8fa7357e4e9384f0115f47299152, State: Initialized, Role: FOLLOWER
I20260808 07:10:23.314252 9314 consensus_queue.cc:260] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.314252 9315 consensus_queue.cc:260] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.314543 9314 ts_tablet_manager.cc:1428] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:23.314543 9315 ts_tablet_manager.cc:1428] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:23.314680 9296 heartbeater.cc:503] Master 127.18.63.126:43033 was elected leader, sending a full tablet report...
I20260808 07:10:23.314759 8958 heartbeater.cc:503] Master 127.18.63.126:43033 was elected leader, sending a full tablet report...
W20260808 07:10:23.357278 9128 tablet.cc:2369] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:23.446120 8959 tablet.cc:2369] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:23.493489 9322 raft_consensus.cc:482] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:23.493631 9322 raft_consensus.cc:504] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.494014 9322 leader_election.cc:288] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb52d56ad64547f3bd5c1f92efab3e36 (127.18.63.67:43549), 6b09e2c43dc24202bce96a6f732f9a18 (127.18.63.66:43877)
I20260808 07:10:23.497563 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.494377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:36289 (local address 127.18.63.67:43549)
0808 07:10:23.494440 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.494442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.494568 (+ 126us) server_negotiation.cc:407] Connection header received
0808 07:10:23.494772 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.494774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.494785 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.494839 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:23.495070 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.495598 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.496798 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.497165 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.497223 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.497252 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.497434 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.497488 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:23.497640 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.494586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:59749 (local address 127.18.63.66:43877)
0808 07:10:23.494652 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.494654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.494739 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:10:23.494851 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.494853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.494868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.494932 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:23.495149 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.495710 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.496704 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.497174 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.497274 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.497283 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.497515 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.497562 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:10:23.497603 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.494261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:43549 (local address 127.18.63.65:36289)
0808 07:10:23.494431 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.494445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.494671 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.494866 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.494867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.494870 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:23.494976 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.494980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.495694 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.495695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.496705 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.496708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.497263 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.497264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.497371 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.497375 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.497423 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.497491 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":51,"thread_start_us":109,"threads_started":1}
I20260808 07:10:23.497853 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.494497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:43877 (local address 127.18.63.65:59749)
0808 07:10:23.494634 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.494645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.494743 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.494954 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.494955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.494957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:23.495061 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.495064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.495806 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.495807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.496602 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.496605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.497293 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.497294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.497417 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.497422 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.497483 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.497521 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260808 07:10:23.498240 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64cafa44918431d9272de25dacf7b52" candidate_uuid: "672d8fa7357e4e9384f0115f47299152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" is_pre_election: true
I20260808 07:10:23.498263 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64cafa44918431d9272de25dacf7b52" candidate_uuid: "672d8fa7357e4e9384f0115f47299152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b09e2c43dc24202bce96a6f732f9a18" is_pre_election: true
I20260808 07:10:23.498380 9241 raft_consensus.cc:2454] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 672d8fa7357e4e9384f0115f47299152 in term 0.
I20260808 07:10:23.498441 9068 raft_consensus.cc:2454] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 672d8fa7357e4e9384f0115f47299152 in term 0.
I20260808 07:10:23.498709 8819 leader_election.cc:302] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 672d8fa7357e4e9384f0115f47299152, 6b09e2c43dc24202bce96a6f732f9a18; no voters:
I20260808 07:10:23.498850 9322 raft_consensus.cc:2781] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:23.498910 9322 raft_consensus.cc:482] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:23.498936 9322 raft_consensus.cc:3037] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:23.499702 9322 raft_consensus.cc:504] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.499846 9322 leader_election.cc:288] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [CANDIDATE]: Term 1 election: Requested vote from peers bb52d56ad64547f3bd5c1f92efab3e36 (127.18.63.67:43549), 6b09e2c43dc24202bce96a6f732f9a18 (127.18.63.66:43877)
I20260808 07:10:23.499996 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64cafa44918431d9272de25dacf7b52" candidate_uuid: "672d8fa7357e4e9384f0115f47299152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb52d56ad64547f3bd5c1f92efab3e36"
I20260808 07:10:23.500080 9241 raft_consensus.cc:3037] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:23.500082 9068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64cafa44918431d9272de25dacf7b52" candidate_uuid: "672d8fa7357e4e9384f0115f47299152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b09e2c43dc24202bce96a6f732f9a18"
I20260808 07:10:23.500177 9068 raft_consensus.cc:3037] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:23.500910 9241 raft_consensus.cc:2454] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 672d8fa7357e4e9384f0115f47299152 in term 1.
I20260808 07:10:23.501084 8819 leader_election.cc:302] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 672d8fa7357e4e9384f0115f47299152, bb52d56ad64547f3bd5c1f92efab3e36; no voters:
I20260808 07:10:23.501195 9322 raft_consensus.cc:2781] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:23.501192 9068 raft_consensus.cc:2454] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 672d8fa7357e4e9384f0115f47299152 in term 1.
I20260808 07:10:23.501435 9322 raft_consensus.cc:686] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [term 1 LEADER]: Becoming Leader. State: Replica: 672d8fa7357e4e9384f0115f47299152, State: Running, Role: LEADER
I20260808 07:10:23.501571 9322 consensus_queue.cc:237] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } }
I20260808 07:10:23.502552 8717 catalog_manager.cc:5495] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 672d8fa7357e4e9384f0115f47299152 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "672d8fa7357e4e9384f0115f47299152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "672d8fa7357e4e9384f0115f47299152" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 46025 } health_report { overall_health: HEALTHY } } }
W20260808 07:10:23.517267 9297 tablet.cc:2369] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:23.531177 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.528705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.18.63.65:46025)
0808 07:10:23.528795 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.528816 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.528826 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:23.528869 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.528870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.528892 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.528961 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:23.529128 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.529696 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.530453 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.530679 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.530737 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.530876 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:23.530981 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:23.531055 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.531097 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.531116 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 07:10:23.533243 9241 raft_consensus.cc:1261] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 1 FOLLOWER]: Refusing update from remote peer 672d8fa7357e4e9384f0115f47299152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:23.533243 9068 raft_consensus.cc:1261] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 1 FOLLOWER]: Refusing update from remote peer 672d8fa7357e4e9384f0115f47299152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:23.533679 9327 consensus_queue.cc:1035] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb52d56ad64547f3bd5c1f92efab3e36" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:23.533814 9322 consensus_queue.cc:1035] T c64cafa44918431d9272de25dacf7b52 P 672d8fa7357e4e9384f0115f47299152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b09e2c43dc24202bce96a6f732f9a18" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 43877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:23.536396 9338 mvcc.cc:205] Tried to move back new op lower bound from 7316164704389021696 to 7316164704263172096. Current Snapshot: MvccSnapshot[applied={T|T < 7316164704389021696}]
I20260808 07:10:23.536351 9337 mvcc.cc:205] Tried to move back new op lower bound from 7316164704389021696 to 7316164704263172096. Current Snapshot: MvccSnapshot[applied={T|T < 7316164704389021696}]
I20260808 07:10:23.539223 8717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:23.539330 8717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:23.540115 8717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d49297c08e054d678e1bd66abbad9909: Sending DeleteTablet for 3 replicas of tablet c64cafa44918431d9272de25dacf7b52
I20260808 07:10:23.540445 9216 tablet_service.cc:1514] Processing DeleteTablet for tablet c64cafa44918431d9272de25dacf7b52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:23 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:57960
I20260808 07:10:23.540503 9043 tablet_service.cc:1514] Processing DeleteTablet for tablet c64cafa44918431d9272de25dacf7b52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:23 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:57630
I20260808 07:10:23.540741 9349 tablet_replica.cc:330] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: stopping tablet replica
I20260808 07:10:23.540781 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8800
I20260808 07:10:23.540884 9349 raft_consensus.cc:2229] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:23.540987 9349 raft_consensus.cc:2258] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:23.541746 9349 ts_tablet_manager.cc:1905] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:23.543995 9349 ts_tablet_manager.cc:1918] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:23.544068 9349 log.cc:1196] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/c64cafa44918431d9272de25dacf7b52
I20260808 07:10:23.544349 9349 ts_tablet_manager.cc:1939] T c64cafa44918431d9272de25dacf7b52 P bb52d56ad64547f3bd5c1f92efab3e36: Deleting consensus metadata
I20260808 07:10:23.545032 8700 catalog_manager.cc:4841] TS bb52d56ad64547f3bd5c1f92efab3e36 (127.18.63.67:43549): tablet c64cafa44918431d9272de25dacf7b52 (table test-table [id=c3a0ce21059a4b7d96b0564a4b38b70e]) successfully deleted
I20260808 07:10:23.549511 9350 tablet_replica.cc:330] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: stopping tablet replica
I20260808 07:10:23.549675 9350 raft_consensus.cc:2229] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:23.549835 9350 raft_consensus.cc:2258] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:23.550325 9350 ts_tablet_manager.cc:1905] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:23.551856 9350 ts_tablet_manager.cc:1918] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:23.551924 9350 log.cc:1196] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/c64cafa44918431d9272de25dacf7b52
I20260808 07:10:23.552206 9350 ts_tablet_manager.cc:1939] T c64cafa44918431d9272de25dacf7b52 P 6b09e2c43dc24202bce96a6f732f9a18: Deleting consensus metadata
I20260808 07:10:23.552743 8700 catalog_manager.cc:4841] TS 6b09e2c43dc24202bce96a6f732f9a18 (127.18.63.66:43877): tablet c64cafa44918431d9272de25dacf7b52 (table test-table [id=c3a0ce21059a4b7d96b0564a4b38b70e]) successfully deleted
W20260808 07:10:23.554497 8701 connection.cc:535] client connection to 127.18.63.65:46025 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 07:10:23.554610 8701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 07:10:23.554899 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8965
I20260808 07:10:23.558770 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.555049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:46025 (local address 127.0.0.1:55140)
0808 07:10:23.558685 (+ 3636us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.558736 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3610}
W20260808 07:10:23.558960 8701 catalog_manager.cc:4584] TS 672d8fa7357e4e9384f0115f47299152 (127.18.63.65:46025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c64cafa44918431d9272de25dacf7b52: Network error: Client connection negotiation failed: client connection to 127.18.63.65:46025: connect: Connection refused (error 111)
I20260808 07:10:23.567292 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 9137
I20260808 07:10:23.582187 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 8681
2026-08-08T07:10:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2279 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T07:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:23Z Disabled control of system clock
I20260808 07:10:23.617909 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33479
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:23.701874 9355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:23.702108 9355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:23.702170 9355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:23.703346 9355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:23.703390 9355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:23.703403 9355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:23.703416 9355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:23.703428 9355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:23.704749 9355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:23.704954 9355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:23.705209 9355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:23.710184 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.710211 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.710250 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.710629 9355 server_base.cc:1034] running on GCE node
I20260808 07:10:23.710879 9355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:23.711181 9355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:23.712374 9355 hybrid_clock.cc:648] HybridClock initialized: now 1786173023712350 us; error 69 us; skew 500 ppm
I20260808 07:10:23.713752 9355 webserver.cc:460] Webserver started at http://127.18.63.126:35019/ using document root <none> and password file <none>
I20260808 07:10:23.714030 9355 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:23.714103 9355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:23.714258 9355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:23.715345 9355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "d6ac63d52b7946a3b6727d9d56e851cc"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "8cabde95777103230fdfce589d2d7e8b"
server_key_iv: "ca18b383eee16241aca1635e62b4b63a"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.715723 9355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "d6ac63d52b7946a3b6727d9d56e851cc"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "8cabde95777103230fdfce589d2d7e8b"
server_key_iv: "ca18b383eee16241aca1635e62b4b63a"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.717042 9355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:23.717922 9370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.718107 9355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:23.718218 9355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "d6ac63d52b7946a3b6727d9d56e851cc"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "8cabde95777103230fdfce589d2d7e8b"
server_key_iv: "ca18b383eee16241aca1635e62b4b63a"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.718293 9355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:23.739003 9355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:23.739355 9355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:23.739522 9355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:23.743664 9355 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33479
I20260808 07:10:23.743702 9428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33479 every 8 connection(s)
I20260808 07:10:23.744059 9355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:23.745107 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:23.747485 9429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: Bootstrap starting.
I20260808 07:10:23.748106 9429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:23.748409 9429 log.cc:824] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:23.749138 9429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: No bootstrap required, opened a new log
I20260808 07:10:23.750795 9429 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } }
I20260808 07:10:23.750936 9429 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:23.750958 9429 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ac63d52b7946a3b6727d9d56e851cc, State: Initialized, Role: FOLLOWER
I20260808 07:10:23.751166 9429 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } }
I20260808 07:10:23.751236 9429 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:23.751276 9429 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:23.751327 9429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:23.751853 9429 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } }
I20260808 07:10:23.751966 9429 leader_election.cc:302] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6ac63d52b7946a3b6727d9d56e851cc; no voters:
I20260808 07:10:23.752162 9429 leader_election.cc:288] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:23.752264 9434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:23.752414 9429 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:23.752501 9434 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [term 1 LEADER]: Becoming Leader. State: Replica: d6ac63d52b7946a3b6727d9d56e851cc, State: Running, Role: LEADER
I20260808 07:10:23.752615 9434 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } }
I20260808 07:10:23.753123 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 9355
I20260808 07:10:23.753145 9435 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } } }
I20260808 07:10:23.753239 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:23.753232 9435 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:23.753238 9436 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ac63d52b7946a3b6727d9d56e851cc. Latest consensus state: current_term: 1 leader_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ac63d52b7946a3b6727d9d56e851cc" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33479 } } }
I20260808 07:10:23.753509 9436 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:23.753525 18685 external_mini_cluster.cc:1428] Setting key a681f4bf5d5b290925f5e472b70754a1
I20260808 07:10:23.754040 9444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:23.754609 9444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:23.756377 9444 catalog_manager.cc:1347] Generated new cluster ID: e47a39b44a554e60bbb61922ba6dd974
I20260808 07:10:23.756440 9444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:23.759160 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.754011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.18.63.126:33479)
0808 07:10:23.754212 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.754221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.754477 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:10:23.754594 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.754600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.754654 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.754812 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.755028 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.756243 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.757129 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.757618 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.757761 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.758479 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.758484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.758484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:23.758672 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.758706 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:23.758717 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.758791 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.758868 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.758951 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260808 07:10:23.776402 9444 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:23.776948 9444 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:23.781440 9444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: Generated new TSK 0
I20260808 07:10:23.781651 9444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:23.789664 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--builtin_ntp_servers=127.18.63.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:23.875952 9457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:23.876205 9457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:23.876245 9457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:23.877431 9457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:23.877521 9457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:23.877562 9457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:23.878949 9457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:23.879236 9457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:23.879563 9457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:23.886108 9463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.886176 9465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:23.886137 9462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:23.886404 9457 server_base.cc:1034] running on GCE node
I20260808 07:10:23.886628 9457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:23.886883 9457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:23.888072 9457 hybrid_clock.cc:648] HybridClock initialized: now 1786173023888064 us; error 39 us; skew 500 ppm
I20260808 07:10:23.889511 9457 webserver.cc:460] Webserver started at http://127.18.63.65:36343/ using document root <none> and password file <none>
I20260808 07:10:23.889818 9457 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:23.889890 9457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:23.890031 9457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:23.891188 9457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "d01d21ec51414e10bbf84850d999d8e6"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "1929dcbbbf79370d27e2444b0d19606c"
server_key_iv: "65551ae5ac7bef52bc2ec66e3ae11442"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.891582 9457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "d01d21ec51414e10bbf84850d999d8e6"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "1929dcbbbf79370d27e2444b0d19606c"
server_key_iv: "65551ae5ac7bef52bc2ec66e3ae11442"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.893010 9457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:23.893929 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.894117 9457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:23.894229 9457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "d01d21ec51414e10bbf84850d999d8e6"
format_stamp: "Formatted at 2026-08-08 07:10:23 on dist-test-slave-bw88"
server_key: "1929dcbbbf79370d27e2444b0d19606c"
server_key_iv: "65551ae5ac7bef52bc2ec66e3ae11442"
server_key_version: "encryptionkey@0"
I20260808 07:10:23.894308 9457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:23.908878 9457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:23.909204 9457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:23.909441 9457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:23.909693 9457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:23.910033 9457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:23.910090 9457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.910125 9457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:23.910151 9457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:23.916290 9457 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:34617
I20260808 07:10:23.916414 9611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:34617 every 8 connection(s)
I20260808 07:10:23.916762 9457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:23.922472 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.917607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:45165 (local address 127.18.63.126:33479)
0808 07:10:23.917683 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:23.917686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:23.917963 (+ 277us) server_negotiation.cc:407] Connection header received
0808 07:10:23.918493 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:23.918494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:23.918506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:23.918557 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:23.919274 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.919765 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.920881 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.921116 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.921191 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.922244 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:23.922245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:23.922245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:23.922249 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:23.922262 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:23.922266 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:23.922308 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:23.922383 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:10:23.922410 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 07:10:23.922736 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:23.917610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33479 (local address 127.18.63.65:45165)
0808 07:10:23.917832 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:23.917869 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:23.918401 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:23.918622 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:23.918626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:23.918635 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:23.919200 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.919205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.919859 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.919860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.920805 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:23.920808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:23.921202 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:23.921203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:23.921320 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:23.922053 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:23.922053 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:23.922083 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:23.922109 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:23.922311 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:23.922315 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:23.922317 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:23.922373 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:10:23.922458 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260808 07:10:23.923262 9612 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33479
I20260808 07:10:23.923377 9612 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:23.923609 9612 heartbeater.cc:511] Master 127.18.63.126:33479 requested a full tablet report, sending...
I20260808 07:10:23.924149 9389 ts_manager.cc:194] Registered new tserver with Master: d01d21ec51414e10bbf84850d999d8e6 (127.18.63.65:34617)
I20260808 07:10:23.924751 9389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.65:45165
I20260808 07:10:23.926141 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 9457
I20260808 07:10:23.926210 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:23.926373 18685 external_mini_cluster.cc:1428] Setting key 3303f69195531d270dc86e6127334a46
I20260808 07:10:23.927263 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--builtin_ntp_servers=127.18.63.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:24.018716 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:24.018962 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:24.018987 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:24.020193 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:24.020231 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:24.020256 9615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:24.021703 9615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:24.021987 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:24.022346 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:24.028030 9625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.028069 9623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.028079 9615 server_base.cc:1034] running on GCE node
W20260808 07:10:24.028035 9622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.028463 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:24.028682 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:24.029831 9615 hybrid_clock.cc:648] HybridClock initialized: now 1786173024029814 us; error 32 us; skew 500 ppm
I20260808 07:10:24.031176 9615 webserver.cc:460] Webserver started at http://127.18.63.66:45901/ using document root <none> and password file <none>
I20260808 07:10:24.031412 9615 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:24.031456 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:24.031558 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:24.032617 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "b9bd7c6db33544bd8413199b7830e312"
server_key_iv: "ff975539572f80d4f31f63c1b1ef7ec7"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.033007 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "b9bd7c6db33544bd8413199b7830e312"
server_key_iv: "ff975539572f80d4f31f63c1b1ef7ec7"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.034389 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:24.035096 9635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.035280 9615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:24.035373 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "b9bd7c6db33544bd8413199b7830e312"
server_key_iv: "ff975539572f80d4f31f63c1b1ef7ec7"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.035429 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:24.043298 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:24.043552 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:24.043653 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:24.043817 9615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:24.044143 9615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:24.044178 9615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.044234 9615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:24.044265 9615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.050267 9615 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:42467
I20260808 07:10:24.050364 9768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:42467 every 8 connection(s)
I20260808 07:10:24.050678 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:24.053136 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 9615
I20260808 07:10:24.053247 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:24.053479 18685 external_mini_cluster.cc:1428] Setting key 93975647991f6e97ae3933b1521ac938
I20260808 07:10:24.054596 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--builtin_ntp_servers=127.18.63.84:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:10:24.056869 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.051450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:33129 (local address 127.18.63.126:33479)
0808 07:10:24.051514 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.051516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.051781 (+ 265us) server_negotiation.cc:407] Connection header received
0808 07:10:24.052244 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.052246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.052257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.052310 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.053148 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.053864 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.055333 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.055523 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.055582 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.056524 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.056526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.056526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:24.056532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.056549 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:24.056554 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.056618 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.056729 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.056759 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 07:10:24.056990 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.051430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33479 (local address 127.18.63.66:33129)
0808 07:10:24.051634 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.051668 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.052158 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.052363 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.052366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.052374 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:24.053026 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.053032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.053976 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.053977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.055247 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.055250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.055614 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.055615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.055791 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.056348 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:24.056349 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:24.056392 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:24.056422 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:24.056633 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:24.056640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:24.056643 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.056711 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.056795 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260808 07:10:24.057695 9770 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33479
I20260808 07:10:24.057814 9770 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:24.058022 9770 heartbeater.cc:511] Master 127.18.63.126:33479 requested a full tablet report, sending...
I20260808 07:10:24.058437 9389 ts_manager.cc:194] Registered new tserver with Master: 1ef27ff9aadb4605ac0fdcbbaf854b10 (127.18.63.66:42467)
I20260808 07:10:24.058821 9389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.66:33129
W20260808 07:10:24.141148 9774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:24.141386 9774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:24.141409 9774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:24.142554 9774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:24.142596 9774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:24.142621 9774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:24.143950 9774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:24.144213 9774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:24.144515 9774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:24.149796 9786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:24.149824 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.149858 9774 server_base.cc:1034] running on GCE node
W20260808 07:10:24.149818 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:24.150224 9774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:24.150485 9774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:24.151727 9774 hybrid_clock.cc:648] HybridClock initialized: now 1786173024151700 us; error 46 us; skew 500 ppm
I20260808 07:10:24.153199 9774 webserver.cc:460] Webserver started at http://127.18.63.67:39969/ using document root <none> and password file <none>
I20260808 07:10:24.153501 9774 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:24.153630 9774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:24.153797 9774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:24.154883 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "8b9030e069064e48b6dd4072702afd86"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "6aaa079b5148c28eac8bb35c0b4c924f"
server_key_iv: "f83c1b5ea7ea479ebe7f21d4151c4e7e"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.155198 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "8b9030e069064e48b6dd4072702afd86"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "6aaa079b5148c28eac8bb35c0b4c924f"
server_key_iv: "f83c1b5ea7ea479ebe7f21d4151c4e7e"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.156515 9774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:24.157284 9792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.157528 9774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:24.157626 9774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "8b9030e069064e48b6dd4072702afd86"
format_stamp: "Formatted at 2026-08-08 07:10:24 on dist-test-slave-bw88"
server_key: "6aaa079b5148c28eac8bb35c0b4c924f"
server_key_iv: "f83c1b5ea7ea479ebe7f21d4151c4e7e"
server_key_version: "encryptionkey@0"
I20260808 07:10:24.157681 9774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:24.171800 9774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:24.172075 9774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:24.172199 9774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:24.172399 9774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:24.172696 9774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:24.172727 9774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.172751 9774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:24.172763 9774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:24.179273 9774 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46759
I20260808 07:10:24.179454 9941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46759 every 8 connection(s)
I20260808 07:10:24.179687 9774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:24.181447 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 9774
I20260808 07:10:24.181552 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:24.181764 18685 external_mini_cluster.cc:1428] Setting key 40802db17b62e8a486a199762166b865
I20260808 07:10:24.185827 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.180532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:42771 (local address 127.18.63.126:33479)
0808 07:10:24.180602 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.180605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.180865 (+ 260us) server_negotiation.cc:407] Connection header received
0808 07:10:24.181495 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.181496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.181525 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.181619 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.182659 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.183267 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.184318 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.184505 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.184558 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.185570 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.185571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.185571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:24.185582 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.185596 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:24.185600 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.185646 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.185743 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.185767 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 07:10:24.186110 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.180494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:33479 (local address 127.18.63.67:42771)
0808 07:10:24.180699 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.180742 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.181336 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.181720 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.181724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.181733 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:24.182542 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.182550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.183373 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.183374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.184238 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.184241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.184591 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.184592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.184713 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.185240 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:24.185240 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:24.185271 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:24.185295 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:24.185663 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:24.185670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:24.185672 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.185736 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.185827 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260808 07:10:24.186847 9942 heartbeater.cc:347] Connected to a master server at 127.18.63.126:33479
I20260808 07:10:24.187079 9942 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:24.187311 9942 heartbeater.cc:511] Master 127.18.63.126:33479 requested a full tablet report, sending...
I20260808 07:10:24.187759 9389 ts_manager.cc:194] Registered new tserver with Master: 8b9030e069064e48b6dd4072702afd86 (127.18.63.67:46759)
I20260808 07:10:24.188117 9389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.67:42771
I20260808 07:10:24.193791 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:24.198453 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.195305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.18.63.126:33479)
0808 07:10:24.195354 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.195357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.195482 (+ 125us) server_negotiation.cc:407] Connection header received
0808 07:10:24.195584 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.195587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.195605 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.195672 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:24.195860 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.196345 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.197165 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.197429 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.197507 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.198185 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:24.198186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:24.198187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:24.198191 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:24.198205 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:24.198209 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:24.198278 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.198334 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.198361 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:24.200138 9389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:24.201963 9389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:24.208568 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46759 (local address 127.0.0.1:54566)
0808 07:10:24.204967 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.204976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.205030 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.205272 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.205273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.205275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:24.205308 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.205310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206611 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.206612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.207424 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.207426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.208262 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.208263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.208344 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.208359 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.208436 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.208486 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":4,"thread_start_us":62,"threads_started":1}
I20260808 07:10:24.208592 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.18.63.67:46759)
0808 07:10:24.204805 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.204809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.205096 (+ 287us) server_negotiation.cc:407] Connection header received
0808 07:10:24.205145 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.205147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.205165 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.205415 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:24.205487 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206547 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.207525 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.208176 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.208253 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.208347 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.208467 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.208506 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
I20260808 07:10:24.209205 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42467 (local address 127.0.0.1:34128)
0808 07:10:24.204703 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.204745 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.204849 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.205092 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.205096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.205098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:24.205174 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.205176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206554 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.206555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.207766 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.207768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.208948 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.208949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.209013 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.209017 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.209082 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.209125 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":83,"thread_start_us":135,"threads_started":1}
I20260808 07:10:24.209224 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.18.63.66:42467)
0808 07:10:24.204687 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.204692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.204829 (+ 137us) server_negotiation.cc:407] Connection header received
0808 07:10:24.204951 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.204954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.204972 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.205065 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:24.205274 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206474 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.207857 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.208599 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.208946 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.209046 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.209099 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.209139 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260808 07:10:24.209497 9853 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 (DEFAULT_TABLE table=test-table [id=153783837da64e0a8dcf4f85caf8d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:24.209688 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34617 (local address 127.0.0.1:42020)
0808 07:10:24.205228 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.205238 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.205288 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.205541 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.205542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.205544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:24.205590 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.205592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206482 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.206484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.208040 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.208043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.209434 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.209435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.209512 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.209517 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.209582 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.209623 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":52,"thread_start_us":135,"threads_started":1}
I20260808 07:10:24.209741 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.204574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.18.63.65:34617)
0808 07:10:24.204884 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.204886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.205375 (+ 489us) server_negotiation.cc:407] Connection header received
0808 07:10:24.205427 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.205429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.205445 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.205535 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:24.205662 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.206386 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.208127 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.209090 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.209158 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.209238 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.209614 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.209662 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":88,"threads_started":1}
I20260808 07:10:24.209965 9853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7deae4a2d44dc185a50b8ba8f6be98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:24.210052 9690 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 (DEFAULT_TABLE table=test-table [id=153783837da64e0a8dcf4f85caf8d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:24.210333 9690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7deae4a2d44dc185a50b8ba8f6be98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:24.212764 9962 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Bootstrap starting.
I20260808 07:10:24.212780 9963 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Bootstrap starting.
I20260808 07:10:24.213320 9532 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 (DEFAULT_TABLE table=test-table [id=153783837da64e0a8dcf4f85caf8d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:24.213519 9962 tablet_bootstrap.cc:654] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:24.213702 9963 tablet_bootstrap.cc:654] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:24.213719 9532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7deae4a2d44dc185a50b8ba8f6be98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:24.213872 9962 log.cc:824] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:24.214097 9963 log.cc:824] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:24.214917 9962 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: No bootstrap required, opened a new log
I20260808 07:10:24.215000 9962 ts_tablet_manager.cc:1397] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:24.215582 9963 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: No bootstrap required, opened a new log
I20260808 07:10:24.215662 9963 ts_tablet_manager.cc:1397] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:10:24.217259 9967 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Bootstrap starting.
I20260808 07:10:24.217705 9962 raft_consensus.cc:348] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.217880 9962 raft_consensus.cc:374] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:24.217931 9962 raft_consensus.cc:729] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9030e069064e48b6dd4072702afd86, State: Initialized, Role: FOLLOWER
I20260808 07:10:24.218091 9962 consensus_queue.cc:260] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.218214 9967 tablet_bootstrap.cc:654] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:24.218129 9963 raft_consensus.cc:348] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.218299 9963 raft_consensus.cc:374] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:24.218341 9963 raft_consensus.cc:729] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef27ff9aadb4605ac0fdcbbaf854b10, State: Initialized, Role: FOLLOWER
I20260808 07:10:24.218375 9962 ts_tablet_manager.cc:1428] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:10:24.218405 9942 heartbeater.cc:503] Master 127.18.63.126:33479 was elected leader, sending a full tablet report...
I20260808 07:10:24.218463 9963 consensus_queue.cc:260] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.218737 9963 ts_tablet_manager.cc:1428] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:24.218787 9770 heartbeater.cc:503] Master 127.18.63.126:33479 was elected leader, sending a full tablet report...
I20260808 07:10:24.218873 9967 log.cc:824] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:24.219808 9967 tablet_bootstrap.cc:492] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: No bootstrap required, opened a new log
I20260808 07:10:24.219892 9967 ts_tablet_manager.cc:1397] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:10:24.221930 9967 raft_consensus.cc:348] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.222080 9967 raft_consensus.cc:374] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:24.222126 9967 raft_consensus.cc:729] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d01d21ec51414e10bbf84850d999d8e6, State: Initialized, Role: FOLLOWER
I20260808 07:10:24.222262 9967 consensus_queue.cc:260] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.222474 9967 ts_tablet_manager.cc:1428] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:24.222563 9612 heartbeater.cc:503] Master 127.18.63.126:33479 was elected leader, sending a full tablet report...
W20260808 07:10:24.301281 9771 tablet.cc:2369] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:24.417567 9613 tablet.cc:2369] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:24.430308 9943 tablet.cc:2369] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:24.504817 9969 raft_consensus.cc:482] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:24.504984 9969 raft_consensus.cc:504] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.505388 9969 leader_election.cc:288] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ef27ff9aadb4605ac0fdcbbaf854b10 (127.18.63.66:42467), d01d21ec51414e10bbf84850d999d8e6 (127.18.63.65:34617)
I20260808 07:10:24.508533 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.505606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34617 (local address 127.18.63.67:56289)
0808 07:10:24.505677 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.505691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.505747 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.505966 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.505968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.505969 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:24.506059 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.506062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.506816 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.506817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.507640 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.507643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.508246 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.508247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.508325 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.508329 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.508392 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.508438 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 07:10:24.508584 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.505728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:56289 (local address 127.18.63.65:34617)
0808 07:10:24.505782 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.505784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.505797 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:10:24.505834 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.505835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.505849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.505924 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:24.506153 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.506717 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.507734 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.508163 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.508221 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.508228 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.508405 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.508504 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:10:24.508834 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.505712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:47955 (local address 127.18.63.66:42467)
0808 07:10:24.505763 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.505765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.505957 (+ 192us) server_negotiation.cc:407] Connection header received
0808 07:10:24.505990 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.505991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.506000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.506055 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:24.506258 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.506743 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.507934 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.508409 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.508482 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.508488 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.508717 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.508767 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:10:24.508893 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f7deae4a2d44dc185a50b8ba8f6be98" candidate_uuid: "8b9030e069064e48b6dd4072702afd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d01d21ec51414e10bbf84850d999d8e6" is_pre_election: true
I20260808 07:10:24.509066 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.505638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:42467 (local address 127.18.63.67:47955)
0808 07:10:24.505831 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.505842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:24.505915 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:24.506077 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:24.506078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:24.506079 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:24.506178 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.506180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.506857 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.506857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.507835 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:24.507838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.508523 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.508524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:24.508608 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.508613 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:24.508932 (+ 319us) client_negotiation.cc:240] Negotiation successful
0808 07:10:24.508993 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260808 07:10:24.509056 9558 raft_consensus.cc:2454] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b9030e069064e48b6dd4072702afd86 in term 0.
I20260808 07:10:24.509363 9798 leader_election.cc:302] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b9030e069064e48b6dd4072702afd86, d01d21ec51414e10bbf84850d999d8e6; no voters:
I20260808 07:10:24.509516 9969 raft_consensus.cc:2781] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:24.509586 9969 raft_consensus.cc:482] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:24.509642 9969 raft_consensus.cc:3037] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:24.509603 9712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f7deae4a2d44dc185a50b8ba8f6be98" candidate_uuid: "8b9030e069064e48b6dd4072702afd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" is_pre_election: true
I20260808 07:10:24.509771 9712 raft_consensus.cc:2454] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b9030e069064e48b6dd4072702afd86 in term 0.
I20260808 07:10:24.510473 9969 raft_consensus.cc:504] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.510625 9969 leader_election.cc:288] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [CANDIDATE]: Term 1 election: Requested vote from peers 1ef27ff9aadb4605ac0fdcbbaf854b10 (127.18.63.66:42467), d01d21ec51414e10bbf84850d999d8e6 (127.18.63.65:34617)
I20260808 07:10:24.510812 9712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f7deae4a2d44dc185a50b8ba8f6be98" candidate_uuid: "8b9030e069064e48b6dd4072702afd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10"
I20260808 07:10:24.510857 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f7deae4a2d44dc185a50b8ba8f6be98" candidate_uuid: "8b9030e069064e48b6dd4072702afd86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d01d21ec51414e10bbf84850d999d8e6"
I20260808 07:10:24.510910 9712 raft_consensus.cc:3037] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:24.510963 9558 raft_consensus.cc:3037] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:24.511739 9558 raft_consensus.cc:2454] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b9030e069064e48b6dd4072702afd86 in term 1.
I20260808 07:10:24.511750 9712 raft_consensus.cc:2454] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b9030e069064e48b6dd4072702afd86 in term 1.
I20260808 07:10:24.511915 9798 leader_election.cc:302] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b9030e069064e48b6dd4072702afd86, d01d21ec51414e10bbf84850d999d8e6; no voters:
I20260808 07:10:24.512007 9969 raft_consensus.cc:2781] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:24.512176 9969 raft_consensus.cc:686] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 1 LEADER]: Becoming Leader. State: Replica: 8b9030e069064e48b6dd4072702afd86, State: Running, Role: LEADER
I20260808 07:10:24.512290 9969 consensus_queue.cc:237] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } }
I20260808 07:10:24.513222 9389 catalog_manager.cc:5495] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b9030e069064e48b6dd4072702afd86 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "8b9030e069064e48b6dd4072702afd86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b9030e069064e48b6dd4072702afd86" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:24.566175 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.563296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54568 (local address 127.18.63.67:46759)
0808 07:10:24.563352 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:24.563354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:24.563365 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:24.563454 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:24.563455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:24.563478 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:24.563632 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:24.563813 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.564346 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.565209 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:24.565453 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:24.565556 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:24.565749 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:24.565872 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:24.565983 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:24.566011 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 07:10:24.566044 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:10:24.568290 9712 raft_consensus.cc:1261] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 1 FOLLOWER]: Refusing update from remote peer 8b9030e069064e48b6dd4072702afd86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:24.568336 9558 raft_consensus.cc:1261] T 4f7deae4a2d44dc185a50b8ba8f6be98 P d01d21ec51414e10bbf84850d999d8e6 [term 1 FOLLOWER]: Refusing update from remote peer 8b9030e069064e48b6dd4072702afd86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:24.568715 9969 consensus_queue.cc:1035] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef27ff9aadb4605ac0fdcbbaf854b10" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 42467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:24.568869 9975 consensus_queue.cc:1035] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d01d21ec51414e10bbf84850d999d8e6" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:24.571497 9980 mvcc.cc:205] Tried to move back new op lower bound from 7316164708628520960 to 7316164708402925568. Current Snapshot: MvccSnapshot[applied={T|T < 7316164708628520960}]
I20260808 07:10:24.572201 9983 mvcc.cc:205] Tried to move back new op lower bound from 7316164708628520960 to 7316164708402925568. Current Snapshot: MvccSnapshot[applied={T|T < 7316164708628520960}]
I20260808 07:10:24.574262 9387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:24.574363 9387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:24.575241 9387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6ac63d52b7946a3b6727d9d56e851cc: Sending DeleteTablet for 3 replicas of tablet 4f7deae4a2d44dc185a50b8ba8f6be98
I20260808 07:10:24.575642 9853 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:24 UTC) from {username='slave'} at 127.0.0.1:54566
I20260808 07:10:24.575637 9690 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:24 UTC) from {username='slave'} at 127.0.0.1:34128
I20260808 07:10:24.575637 9532 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f7deae4a2d44dc185a50b8ba8f6be98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:24 UTC) from {username='slave'} at 127.0.0.1:42020
I20260808 07:10:24.575933 9992 tablet_replica.cc:330] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: stopping tablet replica
I20260808 07:10:24.576094 9992 raft_consensus.cc:2229] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:24.576162 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 9457
I20260808 07:10:24.576265 9992 raft_consensus.cc:2258] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:24.576763 9992 ts_tablet_manager.cc:1905] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:24.578531 9992 ts_tablet_manager.cc:1918] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:24.578598 9992 log.cc:1196] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/4f7deae4a2d44dc185a50b8ba8f6be98
I20260808 07:10:24.578871 9992 ts_tablet_manager.cc:1939] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 8b9030e069064e48b6dd4072702afd86: Deleting consensus metadata
I20260808 07:10:24.579277 9373 catalog_manager.cc:4841] TS 8b9030e069064e48b6dd4072702afd86 (127.18.63.67:46759): tablet 4f7deae4a2d44dc185a50b8ba8f6be98 (table test-table [id=153783837da64e0a8dcf4f85caf8d747]) successfully deleted
I20260808 07:10:24.585508 9995 tablet_replica.cc:330] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: stopping tablet replica
I20260808 07:10:24.585665 9995 raft_consensus.cc:2229] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:24.585819 9995 raft_consensus.cc:2258] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:24.586336 9995 ts_tablet_manager.cc:1905] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:24.588577 9995 ts_tablet_manager.cc:1918] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:24.588673 9995 log.cc:1196] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/4f7deae4a2d44dc185a50b8ba8f6be98
I20260808 07:10:24.588975 9995 ts_tablet_manager.cc:1939] T 4f7deae4a2d44dc185a50b8ba8f6be98 P 1ef27ff9aadb4605ac0fdcbbaf854b10: Deleting consensus metadata
I20260808 07:10:24.589578 9371 catalog_manager.cc:4841] TS 1ef27ff9aadb4605ac0fdcbbaf854b10 (127.18.63.66:42467): tablet 4f7deae4a2d44dc185a50b8ba8f6be98 (table test-table [id=153783837da64e0a8dcf4f85caf8d747]) successfully deleted
W20260808 07:10:24.590165 9374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:34617 (error 108)
I20260808 07:10:24.590499 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 9615
I20260808 07:10:24.590680 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:24.590538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:34617 (local address 127.0.0.1:42024)
0808 07:10:24.590603 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:24.590646 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:34617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 07:10:24.606950 9374 catalog_manager.cc:4584] TS d01d21ec51414e10bbf84850d999d8e6 (127.18.63.65:34617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f7deae4a2d44dc185a50b8ba8f6be98: Network error: Client connection negotiation failed: client connection to 127.18.63.65:34617: connect: Connection refused (error 111)
I20260808 07:10:24.612919 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 9774
I20260808 07:10:24.620252 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 9355
2026-08-08T07:10:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1032 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:24 dist-test-slave-bw88 krb5kdc[9999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:24 dist-test-slave-bw88 krb5kdc[9999](info): set up 2 sockets
Aug 08 07:10:24 dist-test-slave-bw88 krb5kdc[9999](info): commencing operation
krb5kdc: starting...
W20260808 07:10:25.892158 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:25 dist-test-slave-bw88 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:26.020704 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:39263
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:39263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:26.104993 10015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:26.105283 10015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:26.105367 10015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:26.106576 10015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:26.106650 10015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:26.106674 10015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:26.106694 10015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:26.106725 10015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:26.108260 10015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:39263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:26.108578 10015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:26.108826 10015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:26.114121 10021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.114113 10023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.114113 10027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:26.114202 10015 server_base.cc:1034] running on GCE node
I20260808 07:10:26.114555 10015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:26.114817 10015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:26.115963 10015 hybrid_clock.cc:648] HybridClock initialized: now 1786173026115949 us; error 33 us; skew 500 ppm
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:26.118588 10015 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:26.119081 10015 webserver.cc:460] Webserver started at http://127.18.63.126:34091/ using document root <none> and password file <none>
I20260808 07:10:26.119330 10015 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:26.119376 10015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:26.119550 10015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:26.120666 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "024e7bd810654ae88aa36d6713ff8ebf"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "c972d28ae1263fe9686dd584012574d5"
server_key_iv: "61396556ff02669444111dad532467c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.121059 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "024e7bd810654ae88aa36d6713ff8ebf"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "c972d28ae1263fe9686dd584012574d5"
server_key_iv: "61396556ff02669444111dad532467c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.122644 10015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:26.123441 10036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.123624 10015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.123731 10015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "024e7bd810654ae88aa36d6713ff8ebf"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "c972d28ae1263fe9686dd584012574d5"
server_key_iv: "61396556ff02669444111dad532467c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.123818 10015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:26.132489 10015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:26.134469 10015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:26.134656 10015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:26.139264 10015 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:39263
I20260808 07:10:26.139298 10106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:39263 every 8 connection(s)
I20260808 07:10:26.139670 10015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:26.140666 10107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:26.143165 10107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: Bootstrap starting.
I20260808 07:10:26.143774 10107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:26.144080 10107 log.cc:824] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:26.144887 10107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: No bootstrap required, opened a new log
I20260808 07:10:26.146116 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10015
I20260808 07:10:26.146247 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:26.146445 18685 external_mini_cluster.cc:1428] Setting key e358f8a0cb0c15c34247ffae2b0f5eff
I20260808 07:10:26.147258 10107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } }
I20260808 07:10:26.147404 10107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:26.147428 10107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024e7bd810654ae88aa36d6713ff8ebf, State: Initialized, Role: FOLLOWER
I20260808 07:10:26.147586 10107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } }
I20260808 07:10:26.147683 10107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:26.147719 10107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:26.147826 10107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:26.148963 10107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } }
I20260808 07:10:26.149065 10107 leader_election.cc:302] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 024e7bd810654ae88aa36d6713ff8ebf; no voters:
I20260808 07:10:26.149273 10107 leader_election.cc:288] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:26.149295 10116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:26.149435 10116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [term 1 LEADER]: Becoming Leader. State: Replica: 024e7bd810654ae88aa36d6713ff8ebf, State: Running, Role: LEADER
I20260808 07:10:26.149524 10116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } }
I20260808 07:10:26.149586 10107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:26.149850 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "024e7bd810654ae88aa36d6713ff8ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } } }
I20260808 07:10:26.150017 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:26.150231 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 024e7bd810654ae88aa36d6713ff8ebf. Latest consensus state: current_term: 1 leader_uuid: "024e7bd810654ae88aa36d6713ff8ebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024e7bd810654ae88aa36d6713ff8ebf" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39263 } } }
I20260808 07:10:26.150342 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:26.150651 10125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:26.151228 10125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:26.153285 10125 catalog_manager.cc:1347] Generated new cluster ID: e09fd64b26b04080b95778f1de926a80
I20260808 07:10:26.153371 10125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:26.157895 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.147022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.18.63.126:39263)
0808 07:10:26.147251 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.147259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.147299 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:10:26.147428 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.147436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.147508 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.147668 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:26.148589 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.149888 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.151028 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.151625 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.151735 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.154466 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:26.154470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:26.154470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:26.154483 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:26.156235 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.156640 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.156643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.156644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.156763 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.156991 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.156992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.156992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.157239 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:26.157448 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.157537 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.157684 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":135,"threads_started":1}
I20260808 07:10:26.170715 10125 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:26.171239 10125 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:26.173775 10125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: Generated new TSK 0
I20260808 07:10:26.173959 10125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:10:26.208976 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:26.294199 10141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:26.294454 10141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:26.294481 10141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:26.295560 10141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:26.295598 10141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:26.295624 10141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:26.297254 10141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:26.297466 10141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:26.297643 10141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:26.302848 10150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.302973 10147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.302904 10148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:26.303251 10141 server_base.cc:1034] running on GCE node
I20260808 07:10:26.303463 10141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:26.303701 10141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:26.304877 10141 hybrid_clock.cc:648] HybridClock initialized: now 1786173026304854 us; error 42 us; skew 500 ppm
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:26.307533 10141 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:10:26.308023 10141 webserver.cc:460] Webserver started at http://127.18.63.65:39601/ using document root <none> and password file <none>
I20260808 07:10:26.308272 10141 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:26.308343 10141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:26.308495 10141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:26.309633 10141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "4b1471f690034c19849d6d89f2a8aba1"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "0306b91f7e6398aabc6a580a35dfb2de"
server_key_iv: "67fa6b4e56327312ff310881e5820279"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.310000 10141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "4b1471f690034c19849d6d89f2a8aba1"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "0306b91f7e6398aabc6a580a35dfb2de"
server_key_iv: "67fa6b4e56327312ff310881e5820279"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.311368 10141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:26.312098 10158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.312278 10141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:26.312390 10141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "4b1471f690034c19849d6d89f2a8aba1"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "0306b91f7e6398aabc6a580a35dfb2de"
server_key_iv: "67fa6b4e56327312ff310881e5820279"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.312470 10141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:26.335480 10141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:26.337519 10141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:26.337697 10141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:26.337966 10141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:26.338300 10141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:26.338358 10141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.338402 10141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:26.338433 10141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.345059 10141 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:40061
I20260808 07:10:26.345095 10298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:40061 every 8 connection(s)
I20260808 07:10:26.345477 10141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:26.355361 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10141
I20260808 07:10:26.355504 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:26.355702 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.346396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:59867 (local address 127.18.63.126:39263)
0808 07:10:26.346456 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.346458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.346544 (+ 86us) server_negotiation.cc:407] Connection header received
0808 07:10:26.347101 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.347103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.347118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.347191 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:26.348163 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.348703 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.349835 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.350074 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.350144 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.352512 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:26.352513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:26.352514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:26.352522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:26.354093 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.354475 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.354476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.354477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.354533 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.354841 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.354843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.354843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.354975 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:26.355039 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.355530 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.355633 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 07:10:26.355748 18685 external_mini_cluster.cc:1428] Setting key 292c93355449b280964072201ff598f4
I20260808 07:10:26.355926 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.346251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:39263 (local address 127.18.63.65:59867)
0808 07:10:26.346415 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.346459 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.346987 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.347247 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.347250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.347837 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:26.348048 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.348053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.348801 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.348802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.349754 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.349757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.350152 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.350153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.350312 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.350879 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:26.350888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:26.352404 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:26.354171 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.354178 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.354179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.354398 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.354623 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.354624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.354625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.354740 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.355100 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:26.355140 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:26.355325 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.355468 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.355670 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260808 07:10:26.356666 10299 heartbeater.cc:347] Connected to a master server at 127.18.63.126:39263
I20260808 07:10:26.356788 10299 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:26.356972 10299 heartbeater.cc:511] Master 127.18.63.126:39263 requested a full tablet report, sending...
I20260808 07:10:26.357618 10058 ts_manager.cc:194] Registered new tserver with Master: 4b1471f690034c19849d6d89f2a8aba1 (127.18.63.65:40061)
I20260808 07:10:26.358542 10058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:59867
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:10:26.384233 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:26.471146 10306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:26.471362 10306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:26.471387 10306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:26.472532 10306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:26.472568 10306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:26.472594 10306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:26.474174 10306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:26.474387 10306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:26.474576 10306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:26.479859 10318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.479903 10314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:26.480060 10306 server_base.cc:1034] running on GCE node
W20260808 07:10:26.480103 10315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:26.480304 10306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:26.480562 10306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:26.481721 10306 hybrid_clock.cc:648] HybridClock initialized: now 1786173026481713 us; error 30 us; skew 500 ppm
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:26.490047 10306 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:10:26.490558 10306 webserver.cc:460] Webserver started at http://127.18.63.66:35369/ using document root <none> and password file <none>
I20260808 07:10:26.490816 10306 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:26.490888 10306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:26.491039 10306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:26.492132 10306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "f8f51c217a304fe8af441f8f7d93aad0"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "ab12e2f17dcacf9344a77d64e1656210"
server_key_iv: "4672c170e91a1ae61aec5230a1ba9c72"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.492511 10306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "f8f51c217a304fe8af441f8f7d93aad0"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "ab12e2f17dcacf9344a77d64e1656210"
server_key_iv: "4672c170e91a1ae61aec5230a1ba9c72"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.494016 10306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:26.494880 10326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.495194 10306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:26.495311 10306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "f8f51c217a304fe8af441f8f7d93aad0"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "ab12e2f17dcacf9344a77d64e1656210"
server_key_iv: "4672c170e91a1ae61aec5230a1ba9c72"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.495388 10306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:26.504101 10306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:26.506220 10306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:26.506386 10306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:26.506618 10306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:26.506932 10306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:26.506989 10306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.507028 10306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:26.507066 10306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.513450 10306 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:45995
I20260808 07:10:26.513501 10476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:45995 every 8 connection(s)
I20260808 07:10:26.513808 10306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:26.519906 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10306
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:26.520008 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:26.520217 18685 external_mini_cluster.cc:1428] Setting key 8138c8db57e0e5b96e8d574ecb4f483a
I20260808 07:10:26.523656 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.514533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:59723 (local address 127.18.63.126:39263)
0808 07:10:26.514635 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.514637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.514827 (+ 190us) server_negotiation.cc:407] Connection header received
0808 07:10:26.515432 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.515433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.515447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.515513 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:26.516477 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.517004 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.518114 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.518323 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.518410 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.520579 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:26.520581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:26.520581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:26.520591 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:26.522039 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.522589 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.522590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.522590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.522643 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.522948 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.522950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.522950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.523110 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:26.523187 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.523437 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.523536 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 07:10:26.523679 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.514501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:39263 (local address 127.18.63.66:59723)
0808 07:10:26.514698 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.514743 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.515266 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.515584 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.515588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.516156 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:26.516348 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.516353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.517107 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.517108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.517999 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.518002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.518422 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.518423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.518554 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.518971 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:26.518979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:26.520487 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:26.522133 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.522141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.522143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.522491 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.522709 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.522710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.522710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.522866 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.523183 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:26.523188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:26.523303 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.523392 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.523522 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260808 07:10:26.524253 10477 heartbeater.cc:347] Connected to a master server at 127.18.63.126:39263
I20260808 07:10:26.524361 10477 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:26.524556 10477 heartbeater.cc:511] Master 127.18.63.126:39263 requested a full tablet report, sending...
I20260808 07:10:26.524927 10058 ts_manager.cc:194] Registered new tserver with Master: f8f51c217a304fe8af441f8f7d93aad0 (127.18.63.66:45995)
I20260808 07:10:26.525305 10058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:59723
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:10:26.548858 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:26.635133 10482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:26.635352 10482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:26.635377 10482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:26.636554 10482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:26.636600 10482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:26.636626 10482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:26.638235 10482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:26.638487 10482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:26.638736 10482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:26.644207 10492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.644290 10491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:26.644207 10498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:26.644883 10482 server_base.cc:1034] running on GCE node
I20260808 07:10:26.645143 10482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:26.645455 10482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:26.646651 10482 hybrid_clock.cc:648] HybridClock initialized: now 1786173026646632 us; error 36 us; skew 500 ppm
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:26.649612 10482 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:10:26.650127 10482 webserver.cc:460] Webserver started at http://127.18.63.67:43703/ using document root <none> and password file <none>
I20260808 07:10:26.650460 10482 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:26.650530 10482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:26.650693 10482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:26.651945 10482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "51602595cfb54d2baada24be8b441106"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "e452a0e714e81146a3fe65895579105a"
server_key_iv: "6fa153079a02a835dc9f67f470e853ce"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.652335 10482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "51602595cfb54d2baada24be8b441106"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "e452a0e714e81146a3fe65895579105a"
server_key_iv: "6fa153079a02a835dc9f67f470e853ce"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.653832 10482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:26.654626 10506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.654810 10482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:26.654903 10482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "51602595cfb54d2baada24be8b441106"
format_stamp: "Formatted at 2026-08-08 07:10:26 on dist-test-slave-bw88"
server_key: "e452a0e714e81146a3fe65895579105a"
server_key_iv: "6fa153079a02a835dc9f67f470e853ce"
server_key_version: "encryptionkey@0"
I20260808 07:10:26.654986 10482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:26.680395 10482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:26.682895 10482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:26.683063 10482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:26.683321 10482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:26.683656 10482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:26.683713 10482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.683759 10482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:26.683790 10482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:26.690263 10482 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:46665
I20260808 07:10:26.690310 10641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:46665 every 8 connection(s)
I20260808 07:10:26.690634 10482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:26.695272 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10482
I20260808 07:10:26.695366 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:26.695606 18685 external_mini_cluster.cc:1428] Setting key ce788acd3ec23b6c89d44fa37f533a70
Aug 08 07:10:26 dist-test-slave-bw88 krb5kdc[9999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173026, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:26.700241 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.691411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:40515 (local address 127.18.63.126:39263)
0808 07:10:26.691492 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.691494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.691638 (+ 144us) server_negotiation.cc:407] Connection header received
0808 07:10:26.692136 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.692137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.692147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.692202 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:26.693104 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.693670 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.694650 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.694833 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.694890 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.697533 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:26.697534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:26.697534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:26.697541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:26.698888 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.699394 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.699395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.699396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.699451 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.699715 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.699716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.699716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.699815 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:26.699867 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.700109 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.700176 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 07:10:26.700410 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.691305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:39263 (local address 127.18.63.67:40515)
0808 07:10:26.691486 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.691544 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.692044 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.692258 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.692262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.692829 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:26.693025 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.693030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.693751 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.693752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.694567 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.694569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.694908 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.694909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.695015 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.695618 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:26.695629 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:26.697425 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:26.698996 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.699004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.699006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.699282 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.699514 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:26.699515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:26.699515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:26.699667 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:26.699869 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:26.699874 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:26.699989 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.700072 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.700205 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260808 07:10:26.700891 10642 heartbeater.cc:347] Connected to a master server at 127.18.63.126:39263
I20260808 07:10:26.700986 10642 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:26.701140 10642 heartbeater.cc:511] Master 127.18.63.126:39263 requested a full tablet report, sending...
I20260808 07:10:26.701529 10058 ts_manager.cc:194] Registered new tserver with Master: 51602595cfb54d2baada24be8b441106 (127.18.63.67:46665)
I20260808 07:10:26.701926 10058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:40515
I20260808 07:10:26.707480 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:26.715055 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.708916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.18.63.126:39263)
0808 07:10:26.708976 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.708978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.709116 (+ 138us) server_negotiation.cc:407] Connection header received
0808 07:10:26.709170 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.709171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.709179 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.709228 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:26.709812 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.710273 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.710967 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.711151 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.711204 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.712463 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:26.712465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:26.712465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:26.712471 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:26.713907 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.714287 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.714288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.714288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.714339 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:26.714527 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:26.714528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:26.714528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:26.714672 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:26.714753 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.714883 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.714964 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:26.716825 10058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:26.718519 10058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:26.725291 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.720850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:45995 (local address 127.0.0.1:45662)
0808 07:10:26.721438 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.721447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.721505 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.721841 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.721842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.721844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:26.721904 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.721907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.722858 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.722859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.724047 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.724049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.724945 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.724946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.725040 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.725054 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.725139 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.725214 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":101,"threads_started":1}
I20260808 07:10:26.725517 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.720848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46665 (local address 127.0.0.1:51606)
0808 07:10:26.720961 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.721004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.721066 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.721794 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.721798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.721801 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:26.721905 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.721908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.723281 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.723282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.724296 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.724297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.725246 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.725248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.725314 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.725318 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.725399 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.725453 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":28,"thread_start_us":61,"threads_started":1}
I20260808 07:10:26.725325 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.720983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.18.63.66:45995)
0808 07:10:26.721670 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.721674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.721684 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:26.721723 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.721725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.721741 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.721985 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:26.722068 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.722746 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.724148 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.724848 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.724930 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.724994 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.725180 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.725242 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":128,"threads_started":1}
I20260808 07:10:26.725889 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.721243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.18.63.67:46665)
0808 07:10:26.721441 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.721446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.721458 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:10:26.721507 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.721509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.721527 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.721978 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:26.722065 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.723146 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.724385 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.725141 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.725651 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.725725 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.725754 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.725805 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260808 07:10:26.726409 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.721064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40061 (local address 127.0.0.1:38776)
0808 07:10:26.721471 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.721482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.721545 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.721817 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.721818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.721821 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:26.721904 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.721906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.724377 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.724377 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.725132 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.725135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.726188 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.726189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.726252 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.726256 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.726306 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.726346 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260808 07:10:26.726661 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.721202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.18.63.65:40061)
0808 07:10:26.721407 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.721412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.721557 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:10:26.721613 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.721615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.721632 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.721715 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:26.721977 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.723158 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.725227 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.726002 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.726084 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.726151 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.726510 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.726563 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":112,"threads_started":1}
I20260808 07:10:26.726611 10386 tablet_service.cc:1467] Processing CreateTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e (DEFAULT_TABLE table=test-table [id=4d757fbb1f0f40d59e917d55bfbe880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:26.726601 10567 tablet_service.cc:1467] Processing CreateTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e (DEFAULT_TABLE table=test-table [id=4d757fbb1f0f40d59e917d55bfbe880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:26.727008 10567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094bdb29738b4f96972bd1a4ac55fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:26.727010 10386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094bdb29738b4f96972bd1a4ac55fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:26.729959 10665 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Bootstrap starting.
I20260808 07:10:26.730070 10221 tablet_service.cc:1467] Processing CreateTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e (DEFAULT_TABLE table=test-table [id=4d757fbb1f0f40d59e917d55bfbe880b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:26.730284 10666 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Bootstrap starting.
I20260808 07:10:26.730415 10221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 094bdb29738b4f96972bd1a4ac55fc5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:26.730615 10665 tablet_bootstrap.cc:654] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:26.730953 10666 tablet_bootstrap.cc:654] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:26.731057 10665 log.cc:824] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:26.731513 10666 log.cc:824] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:26.731923 10665 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: No bootstrap required, opened a new log
I20260808 07:10:26.731986 10665 ts_tablet_manager.cc:1397] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:26.732496 10666 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: No bootstrap required, opened a new log
I20260808 07:10:26.732553 10666 ts_tablet_manager.cc:1397] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:26.733566 10665 raft_consensus.cc:348] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.733698 10665 raft_consensus.cc:374] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:26.733719 10665 raft_consensus.cc:729] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8f51c217a304fe8af441f8f7d93aad0, State: Initialized, Role: FOLLOWER
I20260808 07:10:26.733893 10669 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Bootstrap starting.
I20260808 07:10:26.734293 10665 consensus_queue.cc:260] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.734606 10669 tablet_bootstrap.cc:654] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:26.734622 10665 ts_tablet_manager.cc:1428] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:26.734649 10477 heartbeater.cc:503] Master 127.18.63.126:39263 was elected leader, sending a full tablet report...
I20260808 07:10:26.734688 10666 raft_consensus.cc:348] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.734836 10666 raft_consensus.cc:374] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:26.734874 10666 raft_consensus.cc:729] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51602595cfb54d2baada24be8b441106, State: Initialized, Role: FOLLOWER
I20260808 07:10:26.734979 10666 consensus_queue.cc:260] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.735232 10669 log.cc:824] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:26.735278 10666 ts_tablet_manager.cc:1428] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:26.735298 10642 heartbeater.cc:503] Master 127.18.63.126:39263 was elected leader, sending a full tablet report...
I20260808 07:10:26.736308 10669 tablet_bootstrap.cc:492] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: No bootstrap required, opened a new log
I20260808 07:10:26.736402 10669 ts_tablet_manager.cc:1397] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:10:26.737831 10669 raft_consensus.cc:348] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.737978 10669 raft_consensus.cc:374] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:26.738024 10669 raft_consensus.cc:729] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1471f690034c19849d6d89f2a8aba1, State: Initialized, Role: FOLLOWER
I20260808 07:10:26.738143 10669 consensus_queue.cc:260] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.738422 10669 ts_tablet_manager.cc:1428] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:10:26.738451 10299 heartbeater.cc:503] Master 127.18.63.126:39263 was elected leader, sending a full tablet report...
W20260808 07:10:26.764385 10478 tablet.cc:2369] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:26.846304 10301 tablet.cc:2369] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:26.941260 10643 tablet.cc:2369] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:26.982822 10670 raft_consensus.cc:482] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:26.982965 10670 raft_consensus.cc:504] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.983321 10670 leader_election.cc:288] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51602595cfb54d2baada24be8b441106 (127.18.63.67:46665), 4b1471f690034c19849d6d89f2a8aba1 (127.18.63.65:40061)
I20260808 07:10:26.986671 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.983806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:51927 (local address 127.18.63.65:40061)
0808 07:10:26.983889 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.983891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.983901 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:26.983934 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.983935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.983946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.984011 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:26.984240 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.984772 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.985900 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.986261 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.986321 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.986329 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.986541 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.986585 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 07:10:26.986670 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.983556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40061 (local address 127.18.63.66:51927)
0808 07:10:26.983669 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.983683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.983736 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.984039 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.984040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.984043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:26.984145 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.984148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.984869 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.984871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.985758 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.985761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.986354 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.986355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.986463 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.986467 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.986527 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.986576 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 07:10:26.986802 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.983556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:46665 (local address 127.18.63.66:45673)
0808 07:10:26.983786 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:26.983796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:26.983881 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:26.984099 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:26.984099 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:26.984101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:26.984170 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.984171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.984901 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.984902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.985760 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:26.985762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.986490 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.986491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:26.986584 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.986588 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:26.986649 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 07:10:26.986687 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":31,"thread_start_us":126,"threads_started":1}
I20260808 07:10:26.986809 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:26.983865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:45673 (local address 127.18.63.67:46665)
0808 07:10:26.983920 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:26.983922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:26.983944 (+ 22us) server_negotiation.cc:407] Connection header received
0808 07:10:26.983972 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:26.983973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:26.984006 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:26.984072 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:26.984283 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.984784 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.985895 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:26.986391 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:26.986473 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:26.986484 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:26.986673 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 07:10:26.986725 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:10:26.987126 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094bdb29738b4f96972bd1a4ac55fc5e" candidate_uuid: "f8f51c217a304fe8af441f8f7d93aad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b1471f690034c19849d6d89f2a8aba1" is_pre_election: true
I20260808 07:10:26.987255 10591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094bdb29738b4f96972bd1a4ac55fc5e" candidate_uuid: "f8f51c217a304fe8af441f8f7d93aad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51602595cfb54d2baada24be8b441106" is_pre_election: true
I20260808 07:10:26.987329 10243 raft_consensus.cc:2454] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f51c217a304fe8af441f8f7d93aad0 in term 0.
I20260808 07:10:26.987396 10591 raft_consensus.cc:2454] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8f51c217a304fe8af441f8f7d93aad0 in term 0.
I20260808 07:10:26.987643 10330 leader_election.cc:302] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b1471f690034c19849d6d89f2a8aba1, f8f51c217a304fe8af441f8f7d93aad0; no voters:
I20260808 07:10:26.987787 10670 raft_consensus.cc:2781] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:26.987856 10670 raft_consensus.cc:482] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:26.987881 10670 raft_consensus.cc:3037] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:26.988695 10670 raft_consensus.cc:504] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.988835 10670 leader_election.cc:288] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [CANDIDATE]: Term 1 election: Requested vote from peers 51602595cfb54d2baada24be8b441106 (127.18.63.67:46665), 4b1471f690034c19849d6d89f2a8aba1 (127.18.63.65:40061)
I20260808 07:10:26.989012 10591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094bdb29738b4f96972bd1a4ac55fc5e" candidate_uuid: "f8f51c217a304fe8af441f8f7d93aad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51602595cfb54d2baada24be8b441106"
I20260808 07:10:26.989019 10243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "094bdb29738b4f96972bd1a4ac55fc5e" candidate_uuid: "f8f51c217a304fe8af441f8f7d93aad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b1471f690034c19849d6d89f2a8aba1"
I20260808 07:10:26.989101 10591 raft_consensus.cc:3037] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:26.989100 10243 raft_consensus.cc:3037] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:26.989931 10591 raft_consensus.cc:2454] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f51c217a304fe8af441f8f7d93aad0 in term 1.
I20260808 07:10:26.990017 10243 raft_consensus.cc:2454] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8f51c217a304fe8af441f8f7d93aad0 in term 1.
I20260808 07:10:26.990180 10328 leader_election.cc:302] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51602595cfb54d2baada24be8b441106, f8f51c217a304fe8af441f8f7d93aad0; no voters:
I20260808 07:10:26.990288 10670 raft_consensus.cc:2781] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:26.990588 10670 raft_consensus.cc:686] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 1 LEADER]: Becoming Leader. State: Replica: f8f51c217a304fe8af441f8f7d93aad0, State: Running, Role: LEADER
I20260808 07:10:26.990707 10670 consensus_queue.cc:237] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } }
I20260808 07:10:26.991607 10058 catalog_manager.cc:5495] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8f51c217a304fe8af441f8f7d93aad0 (127.18.63.66). New cstate: current_term: 1 leader_uuid: "f8f51c217a304fe8af441f8f7d93aad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8f51c217a304fe8af441f8f7d93aad0" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:27.009697 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.007320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.18.63.66:45995)
0808 07:10:27.007378 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.007381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.007396 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:10:27.007433 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.007433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.007452 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.007515 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:27.007642 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.008146 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.008975 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.009173 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.009228 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.009386 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:27.009482 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:27.009565 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.009610 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.009621 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 07:10:27.011607 10243 raft_consensus.cc:1261] T 094bdb29738b4f96972bd1a4ac55fc5e P 4b1471f690034c19849d6d89f2a8aba1 [term 1 FOLLOWER]: Refusing update from remote peer f8f51c217a304fe8af441f8f7d93aad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:27.011607 10591 raft_consensus.cc:1261] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 1 FOLLOWER]: Refusing update from remote peer f8f51c217a304fe8af441f8f7d93aad0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:27.012009 10676 consensus_queue.cc:1035] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51602595cfb54d2baada24be8b441106" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:27.012151 10670 consensus_queue.cc:1035] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b1471f690034c19849d6d89f2a8aba1" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:27.014976 10679 mvcc.cc:205] Tried to move back new op lower bound from 7316164718636818432 to 7316164718554525696. Current Snapshot: MvccSnapshot[applied={T|T < 7316164718636818432}]
I20260808 07:10:27.015164 10683 mvcc.cc:205] Tried to move back new op lower bound from 7316164718636818432 to 7316164718554525696. Current Snapshot: MvccSnapshot[applied={T|T < 7316164718636818432}]
I20260808 07:10:27.017373 10059 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:27.017481 10059 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:27.018345 10059 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 024e7bd810654ae88aa36d6713ff8ebf: Sending DeleteTablet for 3 replicas of tablet 094bdb29738b4f96972bd1a4ac55fc5e
I20260808 07:10:27.018775 10567 tablet_service.cc:1514] Processing DeleteTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:51606
I20260808 07:10:27.018790 10386 tablet_service.cc:1514] Processing DeleteTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:45662
I20260808 07:10:27.018803 10221 tablet_service.cc:1514] Processing DeleteTablet for tablet 094bdb29738b4f96972bd1a4ac55fc5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:38776
I20260808 07:10:27.019114 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10141
I20260808 07:10:27.023826 10695 tablet_replica.cc:330] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: stopping tablet replica
I20260808 07:10:27.023943 10695 raft_consensus.cc:2229] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:27.024088 10695 raft_consensus.cc:2258] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:27.024489 10695 ts_tablet_manager.cc:1905] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:27.026029 10695 ts_tablet_manager.cc:1918] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:27.026146 10695 log.cc:1196] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/094bdb29738b4f96972bd1a4ac55fc5e
I20260808 07:10:27.026403 10695 ts_tablet_manager.cc:1939] T 094bdb29738b4f96972bd1a4ac55fc5e P f8f51c217a304fe8af441f8f7d93aad0: Deleting consensus metadata
I20260808 07:10:27.026819 10042 catalog_manager.cc:4841] TS f8f51c217a304fe8af441f8f7d93aad0 (127.18.63.66:45995): tablet 094bdb29738b4f96972bd1a4ac55fc5e (table test-table [id=4d757fbb1f0f40d59e917d55bfbe880b]) successfully deleted
W20260808 07:10:27.030284 10042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:40061 (error 108)
I20260808 07:10:27.030391 10698 tablet_replica.cc:330] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: stopping tablet replica
I20260808 07:10:27.030539 10698 raft_consensus.cc:2229] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:27.030668 10698 raft_consensus.cc:2258] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:27.030956 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10306
I20260808 07:10:27.031210 10698 ts_tablet_manager.cc:1905] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:27.033138 10698 ts_tablet_manager.cc:1918] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:27.033226 10698 log.cc:1196] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/094bdb29738b4f96972bd1a4ac55fc5e
I20260808 07:10:27.033639 10698 ts_tablet_manager.cc:1939] T 094bdb29738b4f96972bd1a4ac55fc5e P 51602595cfb54d2baada24be8b441106: Deleting consensus metadata
I20260808 07:10:27.034154 10039 catalog_manager.cc:4841] TS 51602595cfb54d2baada24be8b441106 (127.18.63.67:46665): tablet 094bdb29738b4f96972bd1a4ac55fc5e (table test-table [id=4d757fbb1f0f40d59e917d55bfbe880b]) successfully deleted
I20260808 07:10:27.036454 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.036291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:40061 (local address 127.0.0.1:38788)
0808 07:10:27.036352 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.036401 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 07:10:27.038568 10042 catalog_manager.cc:4584] TS 4b1471f690034c19849d6d89f2a8aba1 (127.18.63.65:40061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 094bdb29738b4f96972bd1a4ac55fc5e: Network error: Client connection negotiation failed: client connection to 127.18.63.65:40061: connect: Connection refused (error 111)
I20260808 07:10:27.043380 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10482
I20260808 07:10:27.051142 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10015
2026-08-08T07:10:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2434 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T07:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:27Z Disabled control of system clock
I20260808 07:10:27.084041 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37189
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:27.168591 10704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.168859 10704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.168897 10704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.170076 10704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:27.170156 10704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.170177 10704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.170198 10704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:27.170226 10704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:27.171557 10704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:27.171818 10704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.172120 10704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.177107 10710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.177121 10713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.177121 10711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.177718 10704 server_base.cc:1034] running on GCE node
I20260808 07:10:27.177950 10704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.178244 10704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.179414 10704 hybrid_clock.cc:648] HybridClock initialized: now 1786173027179381 us; error 45 us; skew 500 ppm
I20260808 07:10:27.180640 10704 webserver.cc:460] Webserver started at http://127.18.63.126:38339/ using document root <none> and password file <none>
I20260808 07:10:27.180903 10704 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.180974 10704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.181135 10704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.182268 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "766dec175ef8492ca05e8cc062d9f011"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "a05ee120cb03ec47cc46002e6e158cc9"
server_key_iv: "8932cbd1b1b5cdac5b2ccd4a9d9d9021"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.182650 10704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "766dec175ef8492ca05e8cc062d9f011"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "a05ee120cb03ec47cc46002e6e158cc9"
server_key_iv: "8932cbd1b1b5cdac5b2ccd4a9d9d9021"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.184044 10704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:10:27.184782 10719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.185074 10704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:27.185181 10704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "766dec175ef8492ca05e8cc062d9f011"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "a05ee120cb03ec47cc46002e6e158cc9"
server_key_iv: "8932cbd1b1b5cdac5b2ccd4a9d9d9021"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.185259 10704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.193106 10704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.193445 10704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.193599 10704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.197819 10704 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:37189
I20260808 07:10:27.197858 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:37189 every 8 connection(s)
I20260808 07:10:27.198278 10704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:27.198787 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10704
I20260808 07:10:27.198900 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:27.199110 18685 external_mini_cluster.cc:1428] Setting key 8a74cb0ae129c66de66c2a04443fa6e3
I20260808 07:10:27.199416 10790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:27.202447 10790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: Bootstrap starting.
I20260808 07:10:27.203066 10790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:27.203348 10790 log.cc:824] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:27.204210 10790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: No bootstrap required, opened a new log
I20260808 07:10:27.205297 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.199670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.18.63.126:37189)
0808 07:10:27.199906 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.199915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.199957 (+ 42us) server_negotiation.cc:407] Connection header received
0808 07:10:27.200510 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.200517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.200581 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.200735 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.201000 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.202048 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.203035 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.203612 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.203756 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.204563 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.204568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.204568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:27.204737 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.204772 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:27.204782 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.204840 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.204958 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.205052 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260808 07:10:27.206328 10790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } }
I20260808 07:10:27.206483 10790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:27.206537 10790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766dec175ef8492ca05e8cc062d9f011, State: Initialized, Role: FOLLOWER
I20260808 07:10:27.206660 10790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } }
I20260808 07:10:27.206743 10790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:27.206786 10790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:27.206840 10790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:27.207381 10790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } }
I20260808 07:10:27.207501 10790 leader_election.cc:302] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766dec175ef8492ca05e8cc062d9f011; no voters:
I20260808 07:10:27.207695 10790 leader_election.cc:288] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:27.207754 10798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:27.207949 10798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [term 1 LEADER]: Becoming Leader. State: Replica: 766dec175ef8492ca05e8cc062d9f011, State: Running, Role: LEADER
I20260808 07:10:27.207988 10790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:27.208045 10798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } }
I20260808 07:10:27.208437 10800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "766dec175ef8492ca05e8cc062d9f011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } } }
I20260808 07:10:27.208492 10801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766dec175ef8492ca05e8cc062d9f011. Latest consensus state: current_term: 1 leader_uuid: "766dec175ef8492ca05e8cc062d9f011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766dec175ef8492ca05e8cc062d9f011" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37189 } } }
I20260808 07:10:27.208626 10800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:27.208636 10801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:27.209113 10808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:27.209636 10808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:27.211275 10808 catalog_manager.cc:1347] Generated new cluster ID: f1e2661c63764b13b62df8c9804400db
I20260808 07:10:27.211342 10808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:27.216439 10808 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:27.217123 10808 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:27.226439 10808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: Generated new TSK 0
I20260808 07:10:27.226650 10808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:27.232229 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--builtin_ntp_servers=127.18.63.84:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:27.335680 10818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.335929 10818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.335968 10818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.337109 10818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.337177 10818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.337218 10818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:27.338629 10818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:27.338893 10818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.339200 10818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.345242 10828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.345284 10826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.345477 10825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.345785 10818 server_base.cc:1034] running on GCE node
I20260808 07:10:27.346020 10818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.346297 10818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.347487 10818 hybrid_clock.cc:648] HybridClock initialized: now 1786173027347475 us; error 42 us; skew 500 ppm
I20260808 07:10:27.348980 10818 webserver.cc:460] Webserver started at http://127.18.63.65:40193/ using document root <none> and password file <none>
I20260808 07:10:27.349249 10818 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.349318 10818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.349489 10818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.350941 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "38ec0e916a9d4e46955e56b6a9bb8f24"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "382b9665b039147d10322f19da4ca65f"
server_key_iv: "943c5fbab88194c66bddd9af5325125b"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.351364 10818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "38ec0e916a9d4e46955e56b6a9bb8f24"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "382b9665b039147d10322f19da4ca65f"
server_key_iv: "943c5fbab88194c66bddd9af5325125b"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.352952 10818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:27.353835 10837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.354031 10818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:27.354151 10818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "38ec0e916a9d4e46955e56b6a9bb8f24"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "382b9665b039147d10322f19da4ca65f"
server_key_iv: "943c5fbab88194c66bddd9af5325125b"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.354226 10818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.381376 10818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.381738 10818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.381896 10818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.382153 10818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.382529 10818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.382566 10818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.382591 10818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.382633 10818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.388909 10818 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:33635
I20260808 07:10:27.388998 10998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:33635 every 8 connection(s)
I20260808 07:10:27.389361 10818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:27.394817 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.390155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:53537 (local address 127.18.63.126:37189)
0808 07:10:27.390220 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.390223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.390529 (+ 306us) server_negotiation.cc:407] Connection header received
0808 07:10:27.391063 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.391064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.391082 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.391151 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.391802 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.392313 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.393319 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.393562 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.393644 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.394484 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.394485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.394486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:27.394523 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.394536 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:27.394540 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.394605 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.394703 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.394734 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:10:27.395062 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.390160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37189 (local address 127.18.63.65:53537)
0808 07:10:27.390375 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.390408 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.390938 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.391207 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.391210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.391219 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.391705 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.391710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.392413 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.392414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.393211 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.393213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.393655 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.393656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.393793 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.394341 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:27.394342 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.394370 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:27.394392 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.394608 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.394616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.394619 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.394707 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.394809 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260808 07:10:27.395656 10999 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37189
I20260808 07:10:27.395764 10999 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.395936 10999 heartbeater.cc:511] Master 127.18.63.126:37189 requested a full tablet report, sending...
I20260808 07:10:27.396495 10742 ts_manager.cc:194] Registered new tserver with Master: 38ec0e916a9d4e46955e56b6a9bb8f24 (127.18.63.65:33635)
I20260808 07:10:27.397095 10742 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.65:53537
I20260808 07:10:27.399262 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 10818
I20260808 07:10:27.399374 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:27.399570 18685 external_mini_cluster.cc:1428] Setting key 1201bc4f9a133e573a180533f0668c75
I20260808 07:10:27.400475 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--builtin_ntp_servers=127.18.63.84:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:27.483866 11002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.484074 11002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.484097 11002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.485265 11002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.485308 11002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.485349 11002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:27.486755 11002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:27.486956 11002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.487295 11002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.492518 11012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.492530 11010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.492551 11009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.492791 11002 server_base.cc:1034] running on GCE node
I20260808 07:10:27.493088 11002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.493409 11002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.494582 11002 hybrid_clock.cc:648] HybridClock initialized: now 1786173027494559 us; error 35 us; skew 500 ppm
I20260808 07:10:27.495970 11002 webserver.cc:460] Webserver started at http://127.18.63.66:34019/ using document root <none> and password file <none>
I20260808 07:10:27.496233 11002 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.496307 11002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.496469 11002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.497660 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "f7753f4b99334be18e47e970cd37c28b"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "f74dca04a68599a59b2fb8f8658f007f"
server_key_iv: "3a6ee9a2d32d821ca8c492b8045fa065"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.498034 11002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "f7753f4b99334be18e47e970cd37c28b"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "f74dca04a68599a59b2fb8f8658f007f"
server_key_iv: "3a6ee9a2d32d821ca8c492b8045fa065"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.499639 11002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:27.500391 11023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.500582 11002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:27.500697 11002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "f7753f4b99334be18e47e970cd37c28b"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "f74dca04a68599a59b2fb8f8658f007f"
server_key_iv: "3a6ee9a2d32d821ca8c492b8045fa065"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.500777 11002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.521859 11002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.522218 11002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.522385 11002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.522650 11002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.523015 11002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.523074 11002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.523111 11002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.523166 11002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.529269 11002 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:45273
I20260808 07:10:27.529388 11169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:45273 every 8 connection(s)
I20260808 07:10:27.529721 11002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:27.534914 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.530541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:53599 (local address 127.18.63.126:37189)
0808 07:10:27.530587 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.530589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.530845 (+ 256us) server_negotiation.cc:407] Connection header received
0808 07:10:27.531345 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.531347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.531357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.531412 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.532089 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.532576 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.533593 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.533780 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.533838 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.534702 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.534703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.534703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:27.534707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.534721 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:27.534724 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.534767 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.534832 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.534854 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:27.535135 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.530516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37189 (local address 127.18.63.66:53599)
0808 07:10:27.530718 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.530753 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.531261 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.531467 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.531470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.531479 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.531987 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.531991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.532666 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.532667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.533510 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.533512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.533860 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.533861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.533968 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.534566 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:27.534566 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.534595 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:27.534617 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.534767 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.534770 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.534772 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.534817 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.534893 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260808 07:10:27.535969 11170 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37189
I20260808 07:10:27.536067 11170 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.536221 11170 heartbeater.cc:511] Master 127.18.63.126:37189 requested a full tablet report, sending...
I20260808 07:10:27.536554 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11002
I20260808 07:10:27.536688 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:27.536707 10742 ts_manager.cc:194] Registered new tserver with Master: f7753f4b99334be18e47e970cd37c28b (127.18.63.66:45273)
I20260808 07:10:27.536895 18685 external_mini_cluster.cc:1428] Setting key dd67e02e8cafb38fb10592d24fa52a55
I20260808 07:10:27.537200 10742 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.66:53599
I20260808 07:10:27.538139 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--builtin_ntp_servers=127.18.63.84:43885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:27.623319 11175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:27.623533 11175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:27.623555 11175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:27.624619 11175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:27.624651 11175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:27.624675 11175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:27.625967 11175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:37189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:27.626145 11175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:27.626379 11175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:27.631603 11183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:27.631614 11182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.631825 11175 server_base.cc:1034] running on GCE node
W20260808 07:10:27.631614 11186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:27.632151 11175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:27.632421 11175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:27.633608 11175 hybrid_clock.cc:648] HybridClock initialized: now 1786173027633595 us; error 43 us; skew 500 ppm
I20260808 07:10:27.635144 11175 webserver.cc:460] Webserver started at http://127.18.63.67:41515/ using document root <none> and password file <none>
I20260808 07:10:27.635417 11175 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:27.635500 11175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:27.635679 11175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:27.636849 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "49248d7549134380b5578b1ba12863bc"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "802ea360472461dccf06d12e5b57a01a"
server_key_iv: "319e0b8957ec4c092a36e7dedaaa12f6"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.637255 11175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "49248d7549134380b5578b1ba12863bc"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "802ea360472461dccf06d12e5b57a01a"
server_key_iv: "319e0b8957ec4c092a36e7dedaaa12f6"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.638736 11175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:27.639515 11193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.639712 11175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:27.639860 11175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "49248d7549134380b5578b1ba12863bc"
format_stamp: "Formatted at 2026-08-08 07:10:27 on dist-test-slave-bw88"
server_key: "802ea360472461dccf06d12e5b57a01a"
server_key_iv: "319e0b8957ec4c092a36e7dedaaa12f6"
server_key_version: "encryptionkey@0"
I20260808 07:10:27.639952 11175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:27.650643 11175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:27.650987 11175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:27.651160 11175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:27.651427 11175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:27.651793 11175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:27.651854 11175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.651901 11175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:27.651957 11175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:27.658771 11175 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:44651
I20260808 07:10:27.658830 11336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:44651 every 8 connection(s)
I20260808 07:10:27.659189 11175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:27.664100 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11175
I20260808 07:10:27.664199 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:27.664426 18685 external_mini_cluster.cc:1428] Setting key aa04894a6d0e4bf6e52cfb04717d8a30
I20260808 07:10:27.665391 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.659953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:36253 (local address 127.18.63.126:37189)
0808 07:10:27.660004 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.660006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.660351 (+ 345us) server_negotiation.cc:407] Connection header received
0808 07:10:27.661005 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.661007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.661021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.661090 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.661962 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.662512 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.663631 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.663850 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.663939 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.665065 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.665066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.665067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:27.665073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.665091 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:27.665095 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.665159 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.665244 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.665279 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:27.665654 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.659948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:37189 (local address 127.18.63.67:36253)
0808 07:10:27.660200 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.660248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.660879 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.661180 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.661185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.661203 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:27.661845 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.661851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.662625 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.662626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.663513 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.663515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.663964 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.663965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.664122 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.664890 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:27.664891 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:27.664929 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:27.664960 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:27.665162 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:27.665169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:27.665173 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.665245 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.665379 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260808 07:10:27.666240 11340 heartbeater.cc:347] Connected to a master server at 127.18.63.126:37189
I20260808 07:10:27.666365 11340 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:27.666610 11340 heartbeater.cc:511] Master 127.18.63.126:37189 requested a full tablet report, sending...
I20260808 07:10:27.667038 10740 ts_manager.cc:194] Registered new tserver with Master: 49248d7549134380b5578b1ba12863bc (127.18.63.67:44651)
I20260808 07:10:27.667378 10740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.67:36253
I20260808 07:10:27.676709 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:27.681318 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.678220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.18.63.126:37189)
0808 07:10:27.678290 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.678292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.678406 (+ 114us) server_negotiation.cc:407] Connection header received
0808 07:10:27.678487 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.678488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.678499 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.678572 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:27.678760 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.679241 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.680026 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.680223 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.680308 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.681005 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:27.681007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:27.681007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:27.681011 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:27.681026 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:27.681030 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:27.681143 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.681224 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.681236 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:10:27.682883 10740 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:27.684373 10740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:27.690939 11357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:44651 (local address 127.0.0.1:50038)
0808 07:10:27.686898 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.686909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.686965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.687432 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.687433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.687435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:27.687471 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.687472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688461 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.688462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.689922 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.689925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690648 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.690724 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.690736 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.690812 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.690864 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":37,"thread_start_us":71,"threads_started":1}
I20260808 07:10:27.690951 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.18.63.67:44651)
0808 07:10:27.687214 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.687219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.687229 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:27.687268 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.687270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.687287 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.687363 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:27.687550 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688374 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690023 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690553 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690646 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.690739 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.690827 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.690864 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":92,"threads_started":1}
I20260808 07:10:27.691103 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:45273 (local address 127.0.0.1:36244)
0808 07:10:27.686739 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.686776 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.686842 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.687090 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.687092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.687094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:27.687171 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.687173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688579 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.688580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.689823 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.689825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690857 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.690920 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.690924 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.690975 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.691013 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":24,"thread_start_us":53,"threads_started":1}
I20260808 07:10:27.691288 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.18.63.65:33635)
0808 07:10:27.687228 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.687233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.687244 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:27.687286 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.687288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.687304 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.687384 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:27.687667 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688859 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690099 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690787 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690851 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.690928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.691146 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.691191 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260808 07:10:27.691440 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33635 (local address 127.0.0.1:42362)
0808 07:10:27.687022 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.687032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.687082 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.687510 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.687510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.687512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:27.687563 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.687565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688940 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.688941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.690016 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.690018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690973 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.691024 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.691027 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.691073 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.691104 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260808 07:10:27.691460 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.686699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.18.63.66:45273)
0808 07:10:27.686901 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.686907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.686919 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:10:27.686966 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.686968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.686984 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.687084 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:27.687286 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.688482 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.689914 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.690655 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.690730 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.690819 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.691262 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.691308 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260808 07:10:27.691816 11251 tablet_service.cc:1467] Processing CreateTablet for tablet f622a6f8ded34358a46be496ef3b5eda (DEFAULT_TABLE table=test-table [id=79cd20505e864ba69890f29480d11ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:27.692117 11251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f622a6f8ded34358a46be496ef3b5eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:27.692188 11086 tablet_service.cc:1467] Processing CreateTablet for tablet f622a6f8ded34358a46be496ef3b5eda (DEFAULT_TABLE table=test-table [id=79cd20505e864ba69890f29480d11ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:27.692459 11086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f622a6f8ded34358a46be496ef3b5eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:27.694067 10895 tablet_service.cc:1467] Processing CreateTablet for tablet f622a6f8ded34358a46be496ef3b5eda (DEFAULT_TABLE table=test-table [id=79cd20505e864ba69890f29480d11ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:27.694396 10895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f622a6f8ded34358a46be496ef3b5eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:27.695420 11359 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Bootstrap starting.
I20260808 07:10:27.695621 11360 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Bootstrap starting.
I20260808 07:10:27.696173 11359 tablet_bootstrap.cc:654] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:27.696271 11360 tablet_bootstrap.cc:654] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:27.696571 11359 log.cc:824] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:27.696611 11360 log.cc:824] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:27.697588 11359 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: No bootstrap required, opened a new log
I20260808 07:10:27.697659 11359 ts_tablet_manager.cc:1397] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 07:10:27.697727 11363 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: Bootstrap starting.
I20260808 07:10:27.698050 11360 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: No bootstrap required, opened a new log
I20260808 07:10:27.698118 11360 ts_tablet_manager.cc:1397] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 07:10:27.698359 11363 tablet_bootstrap.cc:654] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:27.698732 11363 log.cc:824] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:27.699940 11363 tablet_bootstrap.cc:492] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: No bootstrap required, opened a new log
I20260808 07:10:27.700012 11363 ts_tablet_manager.cc:1397] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:27.700573 11360 raft_consensus.cc:348] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.700733 11360 raft_consensus.cc:374] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:27.700776 11360 raft_consensus.cc:729] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7753f4b99334be18e47e970cd37c28b, State: Initialized, Role: FOLLOWER
I20260808 07:10:27.700647 11359 raft_consensus.cc:348] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.700832 11359 raft_consensus.cc:374] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:27.700882 11359 raft_consensus.cc:729] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49248d7549134380b5578b1ba12863bc, State: Initialized, Role: FOLLOWER
I20260808 07:10:27.700909 11360 consensus_queue.cc:260] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.701006 11359 consensus_queue.cc:260] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.701201 11360 ts_tablet_manager.cc:1428] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:27.701295 11359 ts_tablet_manager.cc:1428] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 07:10:27.701351 11170 heartbeater.cc:503] Master 127.18.63.126:37189 was elected leader, sending a full tablet report...
I20260808 07:10:27.701431 11340 heartbeater.cc:503] Master 127.18.63.126:37189 was elected leader, sending a full tablet report...
I20260808 07:10:27.702653 11363 raft_consensus.cc:348] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.702798 11363 raft_consensus.cc:374] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:27.702822 11363 raft_consensus.cc:729] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ec0e916a9d4e46955e56b6a9bb8f24, State: Initialized, Role: FOLLOWER
I20260808 07:10:27.702922 11363 consensus_queue.cc:260] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.703176 11363 ts_tablet_manager.cc:1428] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 07:10:27.703239 10999 heartbeater.cc:503] Master 127.18.63.126:37189 was elected leader, sending a full tablet report...
W20260808 07:10:27.780354 11171 tablet.cc:2369] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:27.859606 11367 raft_consensus.cc:482] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:27.859745 11367 raft_consensus.cc:504] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.860195 11367 leader_election.cc:288] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49248d7549134380b5578b1ba12863bc (127.18.63.67:44651), f7753f4b99334be18e47e970cd37c28b (127.18.63.66:45273)
I20260808 07:10:27.863260 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.860569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:35895 (local address 127.18.63.66:45273)
0808 07:10:27.860618 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.860620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.860630 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:27.860706 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.860707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.860720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.860773 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:27.861009 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.861506 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.862497 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.862874 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.862933 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.862940 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.863137 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.863183 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 07:10:27.863274 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.860400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:44651 (local address 127.18.63.65:33643)
0808 07:10:27.860499 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.860515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.860586 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.860733 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.860734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.860736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:27.860818 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.860821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.861581 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.861582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.862397 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.862399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.863025 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.863026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.863080 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.863083 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.863163 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.863198 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":3}
I20260808 07:10:27.863289 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.860508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:33643 (local address 127.18.63.67:44651)
0808 07:10:27.860594 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.860596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.860611 (+ 15us) server_negotiation.cc:407] Connection header received
0808 07:10:27.860649 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.860650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.860660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.860722 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:27.860905 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.861487 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.862543 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.862930 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.863004 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.863012 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.863180 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.863226 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:27.863273 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.860400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:45273 (local address 127.18.63.65:35895)
0808 07:10:27.860559 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.860569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:27.860629 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:27.860815 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:27.860815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:27.860817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:27.860913 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.860915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.861624 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.861625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.862397 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:27.862399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.862968 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.862969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:27.863046 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.863060 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 07:10:27.863103 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 07:10:27.863132 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260808 07:10:27.863662 11110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f622a6f8ded34358a46be496ef3b5eda" candidate_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7753f4b99334be18e47e970cd37c28b" is_pre_election: true
I20260808 07:10:27.863706 11274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f622a6f8ded34358a46be496ef3b5eda" candidate_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49248d7549134380b5578b1ba12863bc" is_pre_election: true
I20260808 07:10:27.863838 11110 raft_consensus.cc:2454] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ec0e916a9d4e46955e56b6a9bb8f24 in term 0.
I20260808 07:10:27.863838 11274 raft_consensus.cc:2454] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ec0e916a9d4e46955e56b6a9bb8f24 in term 0.
I20260808 07:10:27.864087 10841 leader_election.cc:302] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ec0e916a9d4e46955e56b6a9bb8f24, f7753f4b99334be18e47e970cd37c28b; no voters:
I20260808 07:10:27.864236 11367 raft_consensus.cc:2781] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:27.864302 11367 raft_consensus.cc:482] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:27.864321 11367 raft_consensus.cc:3037] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:27.865120 11367 raft_consensus.cc:504] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.865259 11367 leader_election.cc:288] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [CANDIDATE]: Term 1 election: Requested vote from peers 49248d7549134380b5578b1ba12863bc (127.18.63.67:44651), f7753f4b99334be18e47e970cd37c28b (127.18.63.66:45273)
I20260808 07:10:27.865464 11274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f622a6f8ded34358a46be496ef3b5eda" candidate_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49248d7549134380b5578b1ba12863bc"
I20260808 07:10:27.865470 11110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f622a6f8ded34358a46be496ef3b5eda" candidate_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7753f4b99334be18e47e970cd37c28b"
I20260808 07:10:27.865561 11110 raft_consensus.cc:3037] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:27.865563 11274 raft_consensus.cc:3037] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:27.866344 11110 raft_consensus.cc:2454] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ec0e916a9d4e46955e56b6a9bb8f24 in term 1.
I20260808 07:10:27.866411 11274 raft_consensus.cc:2454] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ec0e916a9d4e46955e56b6a9bb8f24 in term 1.
I20260808 07:10:27.866539 10841 leader_election.cc:302] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ec0e916a9d4e46955e56b6a9bb8f24, f7753f4b99334be18e47e970cd37c28b; no voters:
I20260808 07:10:27.866640 11367 raft_consensus.cc:2781] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:27.866806 11367 raft_consensus.cc:686] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [term 1 LEADER]: Becoming Leader. State: Replica: 38ec0e916a9d4e46955e56b6a9bb8f24, State: Running, Role: LEADER
I20260808 07:10:27.866919 11367 consensus_queue.cc:237] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } }
I20260808 07:10:27.867821 10742 catalog_manager.cc:5495] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ec0e916a9d4e46955e56b6a9bb8f24 (127.18.63.65). New cstate: current_term: 1 leader_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ec0e916a9d4e46955e56b6a9bb8f24" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 33635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:27.874996 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.872873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.18.63.65:33635)
0808 07:10:27.872924 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:27.872926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:27.872933 (+ 7us) server_negotiation.cc:407] Connection header received
0808 07:10:27.872959 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:27.872960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:27.872975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:27.873037 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:27.873158 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.873663 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.874355 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:27.874529 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:27.874578 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:27.874714 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:27.874793 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:27.874860 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:27.874901 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 07:10:27.874909 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:10:27.876744 11274 raft_consensus.cc:1261] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 1 FOLLOWER]: Refusing update from remote peer 38ec0e916a9d4e46955e56b6a9bb8f24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:27.876745 11110 raft_consensus.cc:1261] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 1 FOLLOWER]: Refusing update from remote peer 38ec0e916a9d4e46955e56b6a9bb8f24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:27.877162 11367 consensus_queue.cc:1035] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49248d7549134380b5578b1ba12863bc" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:27.877331 11369 consensus_queue.cc:1035] T f622a6f8ded34358a46be496ef3b5eda P 38ec0e916a9d4e46955e56b6a9bb8f24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7753f4b99334be18e47e970cd37c28b" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 45273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:27.879834 11373 mvcc.cc:205] Tried to move back new op lower bound from 7316164722180591616 to 7316164722143600640. Current Snapshot: MvccSnapshot[applied={T|T < 7316164722180591616}]
I20260808 07:10:27.880462 11374 mvcc.cc:205] Tried to move back new op lower bound from 7316164722180591616 to 7316164722143600640. Current Snapshot: MvccSnapshot[applied={T|T < 7316164722180591616}]
I20260808 07:10:27.881054 11376 mvcc.cc:205] Tried to move back new op lower bound from 7316164722180591616 to 7316164722143600640. Current Snapshot: MvccSnapshot[applied={T|T < 7316164722180591616 or (T in {7316164722180591616})}]
I20260808 07:10:27.882652 10742 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:27.882741 10742 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:27.883576 10742 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 766dec175ef8492ca05e8cc062d9f011: Sending DeleteTablet for 3 replicas of tablet f622a6f8ded34358a46be496ef3b5eda
I20260808 07:10:27.883919 11086 tablet_service.cc:1514] Processing DeleteTablet for tablet f622a6f8ded34358a46be496ef3b5eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='slave'} at 127.0.0.1:36244
I20260808 07:10:27.884011 11251 tablet_service.cc:1514] Processing DeleteTablet for tablet f622a6f8ded34358a46be496ef3b5eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='slave'} at 127.0.0.1:50038
I20260808 07:10:27.884220 11384 tablet_replica.cc:330] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: stopping tablet replica
I20260808 07:10:27.884234 10895 tablet_service.cc:1514] Processing DeleteTablet for tablet f622a6f8ded34358a46be496ef3b5eda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:27 UTC) from {username='slave'} at 127.0.0.1:42362
I20260808 07:10:27.884267 11385 tablet_replica.cc:330] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: stopping tablet replica
I20260808 07:10:27.884366 11385 raft_consensus.cc:2229] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:27.884392 11384 raft_consensus.cc:2229] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:27.884476 11385 raft_consensus.cc:2258] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:27.884714 11384 raft_consensus.cc:2258] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:27.884812 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10818
I20260808 07:10:27.884915 11385 ts_tablet_manager.cc:1905] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:27.885823 11384 ts_tablet_manager.cc:1905] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:27.886998 11385 ts_tablet_manager.cc:1918] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:27.887080 11385 log.cc:1196] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/f622a6f8ded34358a46be496ef3b5eda
I20260808 07:10:27.887359 11385 ts_tablet_manager.cc:1939] T f622a6f8ded34358a46be496ef3b5eda P 49248d7549134380b5578b1ba12863bc: Deleting consensus metadata
I20260808 07:10:27.887792 11384 ts_tablet_manager.cc:1918] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:27.887882 11384 log.cc:1196] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/f622a6f8ded34358a46be496ef3b5eda
I20260808 07:10:27.888181 11384 ts_tablet_manager.cc:1939] T f622a6f8ded34358a46be496ef3b5eda P f7753f4b99334be18e47e970cd37c28b: Deleting consensus metadata
I20260808 07:10:27.888573 10723 catalog_manager.cc:4841] TS f7753f4b99334be18e47e970cd37c28b (127.18.63.66:45273): tablet f622a6f8ded34358a46be496ef3b5eda (table test-table [id=79cd20505e864ba69890f29480d11ba4]) successfully deleted
I20260808 07:10:27.889911 10720 catalog_manager.cc:4841] TS 49248d7549134380b5578b1ba12863bc (127.18.63.67:44651): tablet f622a6f8ded34358a46be496ef3b5eda (table test-table [id=79cd20505e864ba69890f29480d11ba4]) successfully deleted
W20260808 07:10:27.895491 10720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:33635 (error 108)
I20260808 07:10:27.895730 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11002
I20260808 07:10:27.897099 11349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:27.895847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:33635 (local address 127.0.0.1:42390)
0808 07:10:27.897016 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:27.897061 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:33635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1143}
W20260808 07:10:27.900390 10720 catalog_manager.cc:4584] TS 38ec0e916a9d4e46955e56b6a9bb8f24 (127.18.63.65:33635): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f622a6f8ded34358a46be496ef3b5eda: Network error: Client connection negotiation failed: client connection to 127.18.63.65:33635: connect: Connection refused (error 111)
I20260808 07:10:27.904681 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11175
I20260808 07:10:27.912919 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 10704
2026-08-08T07:10:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (863 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:27 dist-test-slave-bw88 krb5kdc[11391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:27 dist-test-slave-bw88 krb5kdc[11391](info): set up 2 sockets
Aug 08 07:10:27 dist-test-slave-bw88 krb5kdc[11391](info): commencing operation
krb5kdc: starting...
W20260808 07:10:29.195032 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:29.323812 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46357
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:46357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.407177 11410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.407428 11410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.407465 11410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.408638 11410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:29.408708 11410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.408730 11410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.408751 11410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:29.408782 11410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:29.410257 11410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:46357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:29.410542 11410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.410779 11410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.415944 11419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.416018 11417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.415943 11415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.416087 11410 server_base.cc:1034] running on GCE node
I20260808 07:10:29.416452 11410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.416755 11410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.417923 11410 hybrid_clock.cc:648] HybridClock initialized: now 1786173029417910 us; error 48 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.420614 11410 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:29.421087 11410 webserver.cc:460] Webserver started at http://127.18.63.126:34659/ using document root <none> and password file <none>
I20260808 07:10:29.421326 11410 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.421446 11410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.421607 11410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.422667 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "8c13f4688a244d58b941cfacc49c56f0"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "abb4d43797ad69acbaa64539e5bc0405"
server_key_iv: "951c0710bc690d177affe9a66d924a6b"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.423043 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "8c13f4688a244d58b941cfacc49c56f0"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "abb4d43797ad69acbaa64539e5bc0405"
server_key_iv: "951c0710bc690d177affe9a66d924a6b"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.424367 11410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:29.425117 11429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.425299 11410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:29.425432 11410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8c13f4688a244d58b941cfacc49c56f0"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "abb4d43797ad69acbaa64539e5bc0405"
server_key_iv: "951c0710bc690d177affe9a66d924a6b"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.425508 11410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.435892 11410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.438251 11410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.438416 11410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.442423 11410 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:46357
I20260808 07:10:29.442458 11506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:46357 every 8 connection(s)
I20260808 07:10:29.442831 11410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:29.443681 11507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:29.446010 11507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: Bootstrap starting.
I20260808 07:10:29.446557 11507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:29.446794 11507 log.cc:824] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:29.447628 11507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: No bootstrap required, opened a new log
I20260808 07:10:29.449218 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11410
I20260808 07:10:29.449354 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:29.449570 18685 external_mini_cluster.cc:1428] Setting key 819efe1dbd874386908c6f13cf962e2f
I20260808 07:10:29.449496 11507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } }
I20260808 07:10:29.449663 11507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:29.449713 11507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c13f4688a244d58b941cfacc49c56f0, State: Initialized, Role: FOLLOWER
I20260808 07:10:29.449885 11507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } }
I20260808 07:10:29.449994 11507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:29.450059 11507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:29.450124 11507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:29.450793 11507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } }
I20260808 07:10:29.450941 11507 leader_election.cc:302] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c13f4688a244d58b941cfacc49c56f0; no voters:
I20260808 07:10:29.451125 11507 leader_election.cc:288] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:29.451146 11513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:29.451360 11513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c13f4688a244d58b941cfacc49c56f0, State: Running, Role: LEADER
I20260808 07:10:29.451398 11507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:29.451514 11513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } }
I20260808 07:10:29.453325 11517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c13f4688a244d58b941cfacc49c56f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } } }
I20260808 07:10:29.453469 11517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:29.453696 11514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c13f4688a244d58b941cfacc49c56f0. Latest consensus state: current_term: 1 leader_uuid: "8c13f4688a244d58b941cfacc49c56f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c13f4688a244d58b941cfacc49c56f0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46357 } } }
I20260808 07:10:29.453812 11514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0 [sys.catalog]: This master's current role is: LEADER
W20260808 07:10:29.454897 11535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:29.455425 11535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:29.455498 11525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:29.455693 11525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:29.457471 11525 catalog_manager.cc:1347] Generated new cluster ID: ff75eddaa6d3427d9e1b670886592ef7
I20260808 07:10:29.457525 11525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:29.461647 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.450088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.18.63.126:46357)
0808 07:10:29.450290 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.450299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.450364 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:10:29.450707 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.450713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.450759 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.450864 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.451711 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.452680 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.453939 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.454574 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.454689 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.457680 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.457684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.457685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.457697 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.459786 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.460181 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.460185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.460185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.460309 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.460519 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.460521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.460521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.460848 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.461011 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.461115 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.461299 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260808 07:10:29.465381 11525 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:29.466106 11525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:29.471999 11525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: Generated new TSK 0
I20260808 07:10:29.472164 11525 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
Entry for principal kudu/127.18.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab.
I20260808 07:10:29.503049 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.589416 11541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.589672 11541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.589715 11541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.590865 11541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.590955 11541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.591004 11541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:29.592569 11541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.65.keytab
--principal=kudu/127.18.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:29.592870 11541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.593132 11541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.598496 11554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.598548 11552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.598512 11551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.598948 11541 server_base.cc:1034] running on GCE node
I20260808 07:10:29.599139 11541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.599335 11541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.600468 11541 hybrid_clock.cc:648] HybridClock initialized: now 1786173029600455 us; error 31 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.603139 11541 init.cc:377] Logged in from keytab as kudu/127.18.63.65@KRBTEST.COM (short username kudu)
I20260808 07:10:29.603629 11541 webserver.cc:460] Webserver started at http://127.18.63.65:35029/ using document root <none> and password file <none>
I20260808 07:10:29.603892 11541 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.603963 11541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.604112 11541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.605199 11541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c66dace617ffa8bb361f3e78b8882517"
server_key_iv: "fa94b1f818b9cdb59e681801d3de13a1"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.605616 11541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c66dace617ffa8bb361f3e78b8882517"
server_key_iv: "fa94b1f818b9cdb59e681801d3de13a1"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.606982 11541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:29.607736 11561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.607910 11541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:29.608021 11541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c66dace617ffa8bb361f3e78b8882517"
server_key_iv: "fa94b1f818b9cdb59e681801d3de13a1"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.608098 11541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.632040 11541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.634153 11541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.634342 11541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.634611 11541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.634951 11541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.635008 11541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.635042 11541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.635066 11541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.641441 11541 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:37545
I20260808 07:10:29.641500 11721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:37545 every 8 connection(s)
I20260808 07:10:29.641865 11541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.65@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:29.649492 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11541
I20260808 07:10:29.649609 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:29.649893 18685 external_mini_cluster.cc:1428] Setting key ec4786cc3dd582911c35145292a20f3d
I20260808 07:10:29.651875 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.642710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:46737 (local address 127.18.63.126:46357)
0808 07:10:29.642807 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.642810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.642985 (+ 175us) server_negotiation.cc:407] Connection header received
0808 07:10:29.643538 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.643539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.643558 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.643634 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.644615 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.645143 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.646206 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.646421 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.646522 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.648703 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.648705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.648705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.648713 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.650329 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.650832 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.650833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.650834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.650904 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.651165 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.651165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.651166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.651319 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.651393 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.651640 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.651741 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 07:10:29.651886 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.642619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46357 (local address 127.18.63.65:46737)
0808 07:10:29.642842 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.642887 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.643425 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.643712 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.643716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.644289 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.644499 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.644507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.645251 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.645252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.646104 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.646106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.646536 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.646538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.646667 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.647198 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.647207 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.648600 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.650406 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.650414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.650417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.650743 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.650992 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.650992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.650993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.651097 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.651386 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.651391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.651510 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.651595 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.651722 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":115,"threads_started":1}
I20260808 07:10:29.652594 11722 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46357
I20260808 07:10:29.652716 11722 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.652935 11722 heartbeater.cc:511] Master 127.18.63.126:46357 requested a full tablet report, sending...
I20260808 07:10:29.653599 11451 ts_manager.cc:194] Registered new tserver with Master: fc47b28f5d4b48c9ac8ea4ec2e5b572c (127.18.63.65:37545)
I20260808 07:10:29.654296 11451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.65@KRBTEST.COM'} at 127.18.63.65:46737
WARNING: no policy specified for kudu/127.18.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
Entry for principal kudu/127.18.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab.
I20260808 07:10:29.679981 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.765817 11727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.766052 11727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.766095 11727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.767244 11727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.767314 11727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.767361 11727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:29.768841 11727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.66.keytab
--principal=kudu/127.18.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:29.769104 11727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.769387 11727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.774663 11738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.774668 11740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.774740 11737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.775005 11727 server_base.cc:1034] running on GCE node
I20260808 07:10:29.775364 11727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.775575 11727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.776731 11727 hybrid_clock.cc:648] HybridClock initialized: now 1786173029776698 us; error 49 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.779592 11727 init.cc:377] Logged in from keytab as kudu/127.18.63.66@KRBTEST.COM (short username kudu)
I20260808 07:10:29.780126 11727 webserver.cc:460] Webserver started at http://127.18.63.66:42505/ using document root <none> and password file <none>
I20260808 07:10:29.780364 11727 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.780449 11727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.780599 11727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.781795 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "e991cd1f76c84d3887b3102b779c9576"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c47c97fc0013b11cc001f62974706c82"
server_key_iv: "eacb9182549549c8646e6cead3d5e08a"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.782191 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "e991cd1f76c84d3887b3102b779c9576"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c47c97fc0013b11cc001f62974706c82"
server_key_iv: "eacb9182549549c8646e6cead3d5e08a"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.783638 11727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:29.784369 11749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.784543 11727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:29.784662 11727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "e991cd1f76c84d3887b3102b779c9576"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "c47c97fc0013b11cc001f62974706c82"
server_key_iv: "eacb9182549549c8646e6cead3d5e08a"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.784749 11727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.799286 11727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.801879 11727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.802060 11727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.802311 11727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.802660 11727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.802721 11727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.802768 11727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.802806 11727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.809852 11727 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:40541
I20260808 07:10:29.809914 11896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:40541 every 8 connection(s)
I20260808 07:10:29.810316 11727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:29.815460 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11727
I20260808 07:10:29.815613 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:29.815862 18685 external_mini_cluster.cc:1428] Setting key ee56bdd62a399b36ea2bdc035e5a46a8
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.66@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:29.821103 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.811072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46357 (local address 127.18.63.66:35001)
0808 07:10:29.811236 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.811283 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.811830 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.812156 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.812160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.812786 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.813012 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.813018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.813868 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.813869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.814707 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.814709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.815152 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.815154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.815294 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.815919 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.815931 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.817617 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.819717 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.819724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.819726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.819945 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.820177 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.820177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.820178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.820277 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.820608 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.820613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.820736 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.820820 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.820939 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260808 07:10:29.821120 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.811118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:35001 (local address 127.18.63.126:46357)
0808 07:10:29.811169 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.811171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.811384 (+ 213us) server_negotiation.cc:407] Connection header received
0808 07:10:29.811995 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.811997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.812011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.812079 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.813126 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.813745 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.814829 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.815037 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.815138 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.817723 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.817724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.817724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.817733 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.819609 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.820025 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.820026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.820027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.820098 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.820348 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.820349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.820350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.820521 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.820606 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.820872 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.820974 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:29.821662 11897 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46357
I20260808 07:10:29.821790 11897 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.821979 11897 heartbeater.cc:511] Master 127.18.63.126:46357 requested a full tablet report, sending...
I20260808 07:10:29.822456 11451 ts_manager.cc:194] Registered new tserver with Master: e991cd1f76c84d3887b3102b779c9576 (127.18.63.66:40541)
I20260808 07:10:29.823015 11451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.66@KRBTEST.COM'} at 127.18.63.66:35001
WARNING: no policy specified for kudu/127.18.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
Entry for principal kudu/127.18.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab.
I20260808 07:10:29.844977 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:29.929786 11904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:29.929984 11904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:29.930008 11904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:29.931130 11904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:29.931172 11904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:29.931224 11904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:29.932684 11904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.67.keytab
--principal=kudu/127.18.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:46357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:29.932886 11904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:29.933126 11904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:29.938264 11909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:29.938257 11912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.938364 11904 server_base.cc:1034] running on GCE node
W20260808 07:10:29.938256 11910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:29.938755 11904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:29.939007 11904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:29.940210 11904 hybrid_clock.cc:648] HybridClock initialized: now 1786173029940193 us; error 48 us; skew 500 ppm
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:29.942828 11904 init.cc:377] Logged in from keytab as kudu/127.18.63.67@KRBTEST.COM (short username kudu)
I20260808 07:10:29.943213 11904 webserver.cc:460] Webserver started at http://127.18.63.67:33341/ using document root <none> and password file <none>
I20260808 07:10:29.943387 11904 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:29.943427 11904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:29.943516 11904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:29.944522 11904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "1b3e0401756248278052f580ce5fa815"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "04d05d7400c93e97a0f08935806a9a55"
server_key_iv: "a25f1c02baa18ce2a52839deb5a7a890"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.944831 11904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "1b3e0401756248278052f580ce5fa815"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "04d05d7400c93e97a0f08935806a9a55"
server_key_iv: "a25f1c02baa18ce2a52839deb5a7a890"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.946787 11904 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 07:10:29.947647 11923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.947806 11904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:29.947906 11904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "1b3e0401756248278052f580ce5fa815"
format_stamp: "Formatted at 2026-08-08 07:10:29 on dist-test-slave-bw88"
server_key: "04d05d7400c93e97a0f08935806a9a55"
server_key_iv: "a25f1c02baa18ce2a52839deb5a7a890"
server_key_version: "encryptionkey@0"
I20260808 07:10:29.947961 11904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:29.958649 11904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:29.960470 11904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:29.960582 11904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:29.960755 11904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:29.961079 11904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:29.961114 11904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.961135 11904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:29.961194 11904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:29.967176 11904 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:33433
I20260808 07:10:29.967257 12083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:33433 every 8 connection(s)
I20260808 07:10:29.967571 11904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:29.970283 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 11904
I20260808 07:10:29.970404 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:29.970630 18685 external_mini_cluster.cc:1428] Setting key 2efa775e2ae314bd8adaa31faa40b07f
Aug 08 07:10:29 dist-test-slave-bw88 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173029, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.67@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:29.977279 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.968369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:51071 (local address 127.18.63.126:46357)
0808 07:10:29.968436 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.968438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.968657 (+ 219us) server_negotiation.cc:407] Connection header received
0808 07:10:29.969123 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.969124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.969135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.969188 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.970243 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.970992 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.972273 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.972569 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.972639 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.974906 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.974908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.974908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.974914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.976111 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.976484 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.976485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.976485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.976537 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.976755 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.976755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.976755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.976854 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.976906 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.977152 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.977214 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 07:10:29.977479 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.968346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:46357 (local address 127.18.63.67:51071)
0808 07:10:29.968529 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.968574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.969037 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.969253 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.969256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.969884 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:29.970117 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.970123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.971132 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.971133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.972182 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.972186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.972673 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.972674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.972840 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.973416 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:10:29.973426 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:29.974824 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:29.976187 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.976194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.976196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.976415 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.976600 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:10:29.976601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:10:29.976601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:10:29.976700 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:10:29.976911 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:29.976915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:29.977041 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.977123 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.977242 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260808 07:10:29.977938 12084 heartbeater.cc:347] Connected to a master server at 127.18.63.126:46357
I20260808 07:10:29.978034 12084 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:29.978176 12084 heartbeater.cc:511] Master 127.18.63.126:46357 requested a full tablet report, sending...
I20260808 07:10:29.978490 11451 ts_manager.cc:194] Registered new tserver with Master: 1b3e0401756248278052f580ce5fa815 (127.18.63.67:33433)
I20260808 07:10:29.978818 11451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.63.67@KRBTEST.COM'} at 127.18.63.67:51071
I20260808 07:10:29.982350 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:29.989326 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.983677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.18.63.126:46357)
0808 07:10:29.983739 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.983741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.983816 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:10:29.983861 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.983862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.983869 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.983917 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:29.984431 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.984875 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.985617 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.985791 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.985843 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.986856 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:29.986857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:29.986857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:29.986863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:29.988361 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.988676 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.988677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.988677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.988740 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:29.988909 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:29.988910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:29.988910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:29.989051 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:29.989126 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.989211 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.989268 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:10:29.990868 11451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:29.992420 11451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:29.999063 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.994776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33433 (local address 127.0.0.1:37934)
0808 07:10:29.994962 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.995004 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.995074 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.995633 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.995636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.995639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:29.995713 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.995714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.996677 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.996678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.997773 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.997776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.998713 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.998714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.998808 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.998823 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.998919 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.998977 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260808 07:10:29.999060 12101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.995204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.18.63.67:33433)
0808 07:10:29.995432 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.995436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.995447 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:29.995490 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.995492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.995509 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.995794 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:29.995868 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.996565 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.997867 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.998626 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.998713 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.998779 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.998938 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.998982 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
I20260808 07:10:29.999577 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.994931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.18.63.66:40541)
0808 07:10:29.995168 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.995173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.995475 (+ 302us) server_negotiation.cc:407] Connection header received
0808 07:10:29.995684 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.995688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.995706 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.995763 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:29.996069 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.997126 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.998490 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.999207 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.999275 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.999324 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.999452 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.999492 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":99,"threads_started":1}
I20260808 07:10:29.999596 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.994808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40541 (local address 127.0.0.1:43420)
0808 07:10:29.995385 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.995398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.995570 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.995909 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.995910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.995913 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:29.995969 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.995971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.997241 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.997241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.998399 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.998402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.999280 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.999282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.999354 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.999359 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.999432 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.999480 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":112,"threads_started":1}
I20260808 07:10:29.999804 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.995228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.18.63.65:37545)
0808 07:10:29.995432 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:29.995436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:29.995521 (+ 85us) server_negotiation.cc:407] Connection header received
0808 07:10:29.995586 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:29.995588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:29.995605 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:29.995676 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:29.995893 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.997090 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.998262 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.998965 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.999040 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.999095 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:29.999653 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 07:10:29.999709 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260808 07:10:29.999608 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:29.995047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:37545 (local address 127.0.0.1:52678)
0808 07:10:29.995419 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:29.995428 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:29.995480 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:29.995764 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:29.995765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:29.995767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:29.995799 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.995801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.997202 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.997203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.998173 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:29.998175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:29.999134 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:29.999135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:29.999385 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:29.999390 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:29.999447 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 07:10:29.999495 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":11,"thread_start_us":85,"threads_started":1}
I20260808 07:10:30.000092 11990 tablet_service.cc:1467] Processing CreateTablet for tablet ce4e768cd2d7443d852254ee0ba603cf (DEFAULT_TABLE table=test-table [id=b34c9311748b4b88b3feadf3d79b6f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.000527 11990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4e768cd2d7443d852254ee0ba603cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.000607 11628 tablet_service.cc:1467] Processing CreateTablet for tablet ce4e768cd2d7443d852254ee0ba603cf (DEFAULT_TABLE table=test-table [id=b34c9311748b4b88b3feadf3d79b6f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.000854 11808 tablet_service.cc:1467] Processing CreateTablet for tablet ce4e768cd2d7443d852254ee0ba603cf (DEFAULT_TABLE table=test-table [id=b34c9311748b4b88b3feadf3d79b6f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.000955 11628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4e768cd2d7443d852254ee0ba603cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.001211 11808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4e768cd2d7443d852254ee0ba603cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.003791 12104 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Bootstrap starting.
I20260808 07:10:30.004182 12105 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Bootstrap starting.
I20260808 07:10:30.004654 12104 tablet_bootstrap.cc:654] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.004664 12106 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: Bootstrap starting.
I20260808 07:10:30.005092 12105 tablet_bootstrap.cc:654] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.005389 12104 log.cc:824] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.005427 12106 tablet_bootstrap.cc:654] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.005829 12105 log.cc:824] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.006055 12106 log.cc:824] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.006512 12104 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: No bootstrap required, opened a new log
I20260808 07:10:30.006594 12104 ts_tablet_manager.cc:1397] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 07:10:30.007012 12106 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: No bootstrap required, opened a new log
I20260808 07:10:30.007112 12106 ts_tablet_manager.cc:1397] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 07:10:30.008363 12104 raft_consensus.cc:348] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.008508 12104 raft_consensus.cc:374] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.008530 12104 raft_consensus.cc:729] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e991cd1f76c84d3887b3102b779c9576, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.008652 12104 consensus_queue.cc:260] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.008966 12104 ts_tablet_manager.cc:1428] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 07:10:30.009011 11897 heartbeater.cc:503] Master 127.18.63.126:46357 was elected leader, sending a full tablet report...
I20260808 07:10:30.009291 12106 raft_consensus.cc:348] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.009567 12106 raft_consensus.cc:374] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.009613 12106 raft_consensus.cc:729] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc47b28f5d4b48c9ac8ea4ec2e5b572c, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.009744 12106 consensus_queue.cc:260] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.009872 12105 tablet_bootstrap.cc:492] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: No bootstrap required, opened a new log
I20260808 07:10:30.009948 12105 ts_tablet_manager.cc:1397] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 07:10:30.010074 12106 ts_tablet_manager.cc:1428] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 07:10:30.010125 11722 heartbeater.cc:503] Master 127.18.63.126:46357 was elected leader, sending a full tablet report...
I20260808 07:10:30.011787 12105 raft_consensus.cc:348] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.011940 12105 raft_consensus.cc:374] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.011966 12105 raft_consensus.cc:729] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3e0401756248278052f580ce5fa815, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.012105 12105 consensus_queue.cc:260] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.012423 12105 ts_tablet_manager.cc:1428] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:10:30.012437 12084 heartbeater.cc:503] Master 127.18.63.126:46357 was elected leader, sending a full tablet report...
I20260808 07:10:30.038300 12112 raft_consensus.cc:482] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:30.038419 12112 raft_consensus.cc:504] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.038820 12112 leader_election.cc:288] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b3e0401756248278052f580ce5fa815 (127.18.63.67:33433), e991cd1f76c84d3887b3102b779c9576 (127.18.63.66:40541)
I20260808 07:10:30.041833 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.039028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:40541 (local address 127.18.63.65:35829)
0808 07:10:30.039101 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.039117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.039175 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.039326 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.039327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.039329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.039419 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.039421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.040202 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.040203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.040955 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.040958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.041565 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.041656 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.041660 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.041716 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.041760 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 07:10:30.041888 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.039126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:35829 (local address 127.18.63.66:40541)
0808 07:10:30.039188 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.039190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.039201 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:30.039233 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.039234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.039244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.039319 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.039494 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.040121 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041050 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.041467 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041579 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.041590 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.041749 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.041799 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:10:30.042007 12101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.039203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:49923 (local address 127.18.63.67:33433)
0808 07:10:30.039259 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.039261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.039385 (+ 124us) server_negotiation.cc:407] Connection header received
0808 07:10:30.039512 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.039513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.039520 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.039603 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.039766 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.040216 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041149 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.041624 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041692 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.041701 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.041875 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.041919 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:10:30.042243 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.039028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:33433 (local address 127.18.63.65:49923)
0808 07:10:30.039309 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.039317 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.039401 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.039624 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.039625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.039628 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.039693 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.039694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.040305 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.040306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.041052 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.041054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.041693 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.041694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.041786 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.041790 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.042120 (+ 330us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.042168 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":23,"thread_start_us":128,"threads_started":1}
I20260808 07:10:30.042351 11837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4e768cd2d7443d852254ee0ba603cf" candidate_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e991cd1f76c84d3887b3102b779c9576" is_pre_election: true
I20260808 07:10:30.042518 11837 raft_consensus.cc:2454] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc47b28f5d4b48c9ac8ea4ec2e5b572c in term 0.
I20260808 07:10:30.042578 12019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4e768cd2d7443d852254ee0ba603cf" candidate_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b3e0401756248278052f580ce5fa815" is_pre_election: true
I20260808 07:10:30.042783 12019 raft_consensus.cc:2454] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc47b28f5d4b48c9ac8ea4ec2e5b572c in term 0.
I20260808 07:10:30.042768 11564 leader_election.cc:302] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e991cd1f76c84d3887b3102b779c9576, fc47b28f5d4b48c9ac8ea4ec2e5b572c; no voters:
I20260808 07:10:30.042918 12112 raft_consensus.cc:2781] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:30.042986 12112 raft_consensus.cc:482] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:30.043020 12112 raft_consensus.cc:3037] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.043828 12112 raft_consensus.cc:504] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.043985 12112 leader_election.cc:288] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [CANDIDATE]: Term 1 election: Requested vote from peers 1b3e0401756248278052f580ce5fa815 (127.18.63.67:33433), e991cd1f76c84d3887b3102b779c9576 (127.18.63.66:40541)
I20260808 07:10:30.044133 11837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4e768cd2d7443d852254ee0ba603cf" candidate_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e991cd1f76c84d3887b3102b779c9576"
I20260808 07:10:30.044152 12019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4e768cd2d7443d852254ee0ba603cf" candidate_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b3e0401756248278052f580ce5fa815"
I20260808 07:10:30.044224 12019 raft_consensus.cc:3037] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.044222 11837 raft_consensus.cc:3037] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.044910 12019 raft_consensus.cc:2454] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc47b28f5d4b48c9ac8ea4ec2e5b572c in term 1.
I20260808 07:10:30.044950 11837 raft_consensus.cc:2454] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc47b28f5d4b48c9ac8ea4ec2e5b572c in term 1.
I20260808 07:10:30.045089 11562 leader_election.cc:302] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b3e0401756248278052f580ce5fa815, fc47b28f5d4b48c9ac8ea4ec2e5b572c; no voters:
I20260808 07:10:30.045197 12112 raft_consensus.cc:2781] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:30.045421 12112 raft_consensus.cc:686] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [term 1 LEADER]: Becoming Leader. State: Replica: fc47b28f5d4b48c9ac8ea4ec2e5b572c, State: Running, Role: LEADER
I20260808 07:10:30.045523 12112 consensus_queue.cc:237] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } }
I20260808 07:10:30.046376 11451 catalog_manager.cc:5495] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c reported cstate change: term changed from 0 to 1, leader changed from <none> to fc47b28f5d4b48c9ac8ea4ec2e5b572c (127.18.63.65). New cstate: current_term: 1 leader_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc47b28f5d4b48c9ac8ea4ec2e5b572c" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 37545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:30.058147 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.055706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.18.63.65:37545)
0808 07:10:30.055786 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.055788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.055799 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:30.055839 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.055840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.055860 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.055928 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:30.056066 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.056564 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.057307 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.057598 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.057660 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.057801 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:30.057897 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:30.057960 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.058000 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.058018 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 07:10:30.060019 12019 raft_consensus.cc:1261] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 1 FOLLOWER]: Refusing update from remote peer fc47b28f5d4b48c9ac8ea4ec2e5b572c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:30.060019 11837 raft_consensus.cc:1261] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 1 FOLLOWER]: Refusing update from remote peer fc47b28f5d4b48c9ac8ea4ec2e5b572c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:30.060377 12117 consensus_queue.cc:1035] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e991cd1f76c84d3887b3102b779c9576" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:30.060559 12112 consensus_queue.cc:1035] T ce4e768cd2d7443d852254ee0ba603cf P fc47b28f5d4b48c9ac8ea4ec2e5b572c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b3e0401756248278052f580ce5fa815" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 07:10:30.060925 11898 tablet.cc:2369] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:30.063162 12122 mvcc.cc:205] Tried to move back new op lower bound from 7316164731123040256 to 7316164731067170816. Current Snapshot: MvccSnapshot[applied={T|T < 7316164731123040256}]
I20260808 07:10:30.063395 12125 mvcc.cc:205] Tried to move back new op lower bound from 7316164731123040256 to 7316164731067170816. Current Snapshot: MvccSnapshot[applied={T|T < 7316164731123040256}]
I20260808 07:10:30.063786 12124 mvcc.cc:205] Tried to move back new op lower bound from 7316164731123040256 to 7316164731067170816. Current Snapshot: MvccSnapshot[applied={T|T < 7316164731123040256}]
I20260808 07:10:30.065286 11450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:30.065421 11450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:30.066180 11450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c13f4688a244d58b941cfacc49c56f0: Sending DeleteTablet for 3 replicas of tablet ce4e768cd2d7443d852254ee0ba603cf
I20260808 07:10:30.066562 11808 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4e768cd2d7443d852254ee0ba603cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:43420
I20260808 07:10:30.066573 11628 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4e768cd2d7443d852254ee0ba603cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:52678
I20260808 07:10:30.066593 11990 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4e768cd2d7443d852254ee0ba603cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:30 UTC) from {username='kudu', principal='kudu/127.18.63.126@KRBTEST.COM'} at 127.0.0.1:37934
I20260808 07:10:30.066833 12137 tablet_replica.cc:330] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: stopping tablet replica
I20260808 07:10:30.066934 12137 raft_consensus.cc:2229] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:30.067028 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11541
I20260808 07:10:30.067054 12137 raft_consensus.cc:2258] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:30.067201 12134 tablet_replica.cc:330] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: stopping tablet replica
I20260808 07:10:30.067291 12134 raft_consensus.cc:2229] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:30.067378 12134 raft_consensus.cc:2258] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:30.071729 12137 ts_tablet_manager.cc:1905] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:30.073596 12137 ts_tablet_manager.cc:1918] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:30.073612 12134 ts_tablet_manager.cc:1905] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:30.073689 12137 log.cc:1196] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/ce4e768cd2d7443d852254ee0ba603cf
I20260808 07:10:30.073957 12137 ts_tablet_manager.cc:1939] T ce4e768cd2d7443d852254ee0ba603cf P e991cd1f76c84d3887b3102b779c9576: Deleting consensus metadata
I20260808 07:10:30.074368 11431 catalog_manager.cc:4841] TS e991cd1f76c84d3887b3102b779c9576 (127.18.63.66:40541): tablet ce4e768cd2d7443d852254ee0ba603cf (table test-table [id=b34c9311748b4b88b3feadf3d79b6f8b]) successfully deleted
I20260808 07:10:30.075632 12134 ts_tablet_manager.cc:1918] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:30.075712 12134 log.cc:1196] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/ce4e768cd2d7443d852254ee0ba603cf
I20260808 07:10:30.075995 12134 ts_tablet_manager.cc:1939] T ce4e768cd2d7443d852254ee0ba603cf P 1b3e0401756248278052f580ce5fa815: Deleting consensus metadata
I20260808 07:10:30.076403 11430 catalog_manager.cc:4841] TS 1b3e0401756248278052f580ce5fa815 (127.18.63.67:33433): tablet ce4e768cd2d7443d852254ee0ba603cf (table test-table [id=b34c9311748b4b88b3feadf3d79b6f8b]) successfully deleted
W20260808 07:10:30.080111 11430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:37545 (error 108)
I20260808 07:10:30.080397 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11727
I20260808 07:10:30.080693 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.080564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:37545 (local address 127.0.0.1:52710)
0808 07:10:30.080615 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.080652 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:37545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 07:10:30.081784 11430 catalog_manager.cc:4584] TS fc47b28f5d4b48c9ac8ea4ec2e5b572c (127.18.63.65:37545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce4e768cd2d7443d852254ee0ba603cf: Network error: Client connection negotiation failed: client connection to 127.18.63.65:37545: connect: Connection refused (error 111)
I20260808 07:10:30.090085 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11904
I20260808 07:10:30.097496 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 11410
2026-08-08T07:10:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T07:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:30Z Disabled control of system clock
I20260808 07:10:30.130407 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42125
--webserver_interface=127.18.63.126
--webserver_port=0
--builtin_ntp_servers=127.18.63.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:42125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:30.213713 12144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:30.213930 12144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:30.213953 12144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:30.215188 12144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:30.215234 12144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:30.215247 12144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:30.215260 12144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:30.215272 12144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:30.216629 12144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:42125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:42125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:30.216892 12144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:30.217197 12144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:30.222157 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.222152 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.222194 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.222452 12144 server_base.cc:1034] running on GCE node
I20260808 07:10:30.222693 12144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:30.222987 12144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:30.224169 12144 hybrid_clock.cc:648] HybridClock initialized: now 1786173030224161 us; error 41 us; skew 500 ppm
I20260808 07:10:30.225476 12144 webserver.cc:460] Webserver started at http://127.18.63.126:40367/ using document root <none> and password file <none>
I20260808 07:10:30.225752 12144 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:30.225823 12144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:30.225975 12144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:30.227075 12144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "fa59b02377a14ceab9049a867e166eac"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "45eb265c826401b2321aeafd6f5a6503"
server_key_iv: "3be938a87da529b2193442303d0eb3c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.227447 12144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "fa59b02377a14ceab9049a867e166eac"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "45eb265c826401b2321aeafd6f5a6503"
server_key_iv: "3be938a87da529b2193442303d0eb3c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.228822 12144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:30.229710 12164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.229894 12144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:30.229986 12144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "fa59b02377a14ceab9049a867e166eac"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "45eb265c826401b2321aeafd6f5a6503"
server_key_iv: "3be938a87da529b2193442303d0eb3c6"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.230082 12144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:30.240188 12144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:30.240547 12144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:30.240717 12144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:30.245448 12144 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:42125
I20260808 07:10:30.245489 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:42125 every 8 connection(s)
I20260808 07:10:30.245872 12144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:30.246955 12229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.249331 12229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: Bootstrap starting.
I20260808 07:10:30.250049 12229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.250357 12229 log.cc:824] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.251255 12229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: No bootstrap required, opened a new log
I20260808 07:10:30.252974 12229 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } }
I20260808 07:10:30.253120 12229 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.253144 12229 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa59b02377a14ceab9049a867e166eac, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.253306 12229 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } }
I20260808 07:10:30.253436 12229 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:30.253481 12229 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:30.253530 12229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.254065 12229 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } }
I20260808 07:10:30.254184 12229 leader_election.cc:302] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa59b02377a14ceab9049a867e166eac; no voters:
I20260808 07:10:30.254372 12229 leader_election.cc:288] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:30.254429 12233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:30.254647 12229 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:30.254685 12233 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [term 1 LEADER]: Becoming Leader. State: Replica: fa59b02377a14ceab9049a867e166eac, State: Running, Role: LEADER
I20260808 07:10:30.254863 12233 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } }
I20260808 07:10:30.254966 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 12144
I20260808 07:10:30.255077 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:30.255318 18685 external_mini_cluster.cc:1428] Setting key 6fc10c76a84e2b981830c0d745704f29
I20260808 07:10:30.255286 12234 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa59b02377a14ceab9049a867e166eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } } }
I20260808 07:10:30.255366 12234 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:30.255406 12237 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa59b02377a14ceab9049a867e166eac. Latest consensus state: current_term: 1 leader_uuid: "fa59b02377a14ceab9049a867e166eac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa59b02377a14ceab9049a867e166eac" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 42125 } } }
I20260808 07:10:30.255486 12237 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:30.255673 12240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:30.256311 12240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:30.258088 12240 catalog_manager.cc:1347] Generated new cluster ID: 1466ed8a982840cbb186607e31b9a9c3
I20260808 07:10:30.258194 12240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:30.261811 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.256083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.18.63.126:42125)
0808 07:10:30.256298 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.256307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.256339 (+ 32us) server_negotiation.cc:407] Connection header received
0808 07:10:30.256470 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.256478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.256547 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.256697 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:30.256979 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.258406 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.259515 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.260174 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.260306 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.261028 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.261032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.261033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:30.261269 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.261318 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:30.261365 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.261440 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.261503 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.261588 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260808 07:10:30.263898 12240 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:30.264385 12240 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:30.269296 12240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: Generated new TSK 0
I20260808 07:10:30.269516 12240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:30.274572 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.65:0
--local_ip_for_outbound_sockets=127.18.63.65
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--builtin_ntp_servers=127.18.63.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 07:10:30.361209 12263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:30.361481 12263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:30.361510 12263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:30.362684 12263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:30.362738 12263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:30.362762 12263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.65
I20260808 07:10:30.364137 12263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.63.65
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.65
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:30.364379 12263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:30.364689 12263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:30.370893 12271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.370954 12270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.370873 12274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.371299 12263 server_base.cc:1034] running on GCE node
I20260808 07:10:30.371536 12263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:30.371788 12263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:30.372969 12263 hybrid_clock.cc:648] HybridClock initialized: now 1786173030372953 us; error 31 us; skew 500 ppm
I20260808 07:10:30.374491 12263 webserver.cc:460] Webserver started at http://127.18.63.65:37313/ using document root <none> and password file <none>
I20260808 07:10:30.374773 12263 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:30.374822 12263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:30.374993 12263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:30.376152 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data/instance:
uuid: "7bd2fb3a1f424c5a82285b70504cebcd"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "ca9f2d52a4ee0cbe5d446399f6d0f59b"
server_key_iv: "07e1a61ebcf5050266e3256c7fc7dd37"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.376547 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance:
uuid: "7bd2fb3a1f424c5a82285b70504cebcd"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "ca9f2d52a4ee0cbe5d446399f6d0f59b"
server_key_iv: "07e1a61ebcf5050266e3256c7fc7dd37"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.378090 12263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 07:10:30.378909 12282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.379091 12263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:30.379209 12263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal
uuid: "7bd2fb3a1f424c5a82285b70504cebcd"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "ca9f2d52a4ee0cbe5d446399f6d0f59b"
server_key_iv: "07e1a61ebcf5050266e3256c7fc7dd37"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.379354 12263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:30.388896 12263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:30.389207 12263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:30.389417 12263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:30.389652 12263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:30.389991 12263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:30.390048 12263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.390089 12263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:30.390117 12263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.396337 12263 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.65:43841
I20260808 07:10:30.396391 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.65:43841 every 8 connection(s)
I20260808 07:10:30.396721 12263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/data/info.pb
I20260808 07:10:30.400966 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 12263
I20260808 07:10:30.401098 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-0/wal/instance
I20260808 07:10:30.401355 18685 external_mini_cluster.cc:1428] Setting key e0b507788ec42694776e49b3dcfadfb1
I20260808 07:10:30.402482 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.66:0
--local_ip_for_outbound_sockets=127.18.63.66
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--builtin_ntp_servers=127.18.63.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:10:30.402650 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.397458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.65:43079 (local address 127.18.63.126:42125)
0808 07:10:30.397524 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.397528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.397788 (+ 260us) server_negotiation.cc:407] Connection header received
0808 07:10:30.398347 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.398348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.398364 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.398425 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:30.399115 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.399630 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.400653 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.400952 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.401035 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.402286 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.402288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.402288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:30.402294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.402308 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:30.402313 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.402371 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.402488 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.402516 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:10:30.402868 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.397461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42125 (local address 127.18.63.65:43079)
0808 07:10:30.397662 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.397695 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.398235 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.398497 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.398501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.398511 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:30.399010 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.399015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.399737 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.399738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.400556 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.400558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.401103 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.401104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.401311 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.402103 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:30.402104 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:30.402143 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:30.402173 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:30.402392 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:30.402398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:30.402402 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.402484 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.402615 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260808 07:10:30.403455 12439 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42125
I20260808 07:10:30.403564 12439 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:30.403764 12439 heartbeater.cc:511] Master 127.18.63.126:42125 requested a full tablet report, sending...
I20260808 07:10:30.404338 12188 ts_manager.cc:194] Registered new tserver with Master: 7bd2fb3a1f424c5a82285b70504cebcd (127.18.63.65:43841)
I20260808 07:10:30.404973 12188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.65:43079
W20260808 07:10:30.487802 12448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:30.488037 12448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:30.488078 12448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:30.489210 12448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:30.489277 12448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:30.489320 12448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.66
I20260808 07:10:30.490684 12448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.63.66
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.66
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:30.490957 12448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:30.491290 12448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:30.496701 12457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.496742 12448 server_base.cc:1034] running on GCE node
W20260808 07:10:30.496778 12454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.496713 12455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.497294 12448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:30.497596 12448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:30.498749 12448 hybrid_clock.cc:648] HybridClock initialized: now 1786173030498741 us; error 38 us; skew 500 ppm
I20260808 07:10:30.500037 12448 webserver.cc:460] Webserver started at http://127.18.63.66:45213/ using document root <none> and password file <none>
I20260808 07:10:30.500310 12448 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:30.500384 12448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:30.500538 12448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:30.501675 12448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data/instance:
uuid: "fc93cf34a491447fa3ebf4aae3fc4b15"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "efba8035ebd04de2876d91e162f3bf11"
server_key_iv: "d560e7e2e5faa7b23b3a8601c35bc78e"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.502036 12448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance:
uuid: "fc93cf34a491447fa3ebf4aae3fc4b15"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "efba8035ebd04de2876d91e162f3bf11"
server_key_iv: "d560e7e2e5faa7b23b3a8601c35bc78e"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.503413 12448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:30.504196 12466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.504485 12448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:30.504597 12448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal
uuid: "fc93cf34a491447fa3ebf4aae3fc4b15"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "efba8035ebd04de2876d91e162f3bf11"
server_key_iv: "d560e7e2e5faa7b23b3a8601c35bc78e"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.504671 12448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:30.535698 12448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:30.536072 12448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:30.536242 12448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:30.536525 12448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:30.536895 12448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:30.536953 12448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.536994 12448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:30.537025 12448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.543334 12448 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.66:35187
I20260808 07:10:30.543401 12636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.66:35187 every 8 connection(s)
I20260808 07:10:30.543742 12448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/data/info.pb
I20260808 07:10:30.549325 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 12448
I20260808 07:10:30.549439 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.544606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.66:55367 (local address 127.18.63.126:42125)
0808 07:10:30.544661 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.544662 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.544880 (+ 218us) server_negotiation.cc:407] Connection header received
0808 07:10:30.545447 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.545449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.545461 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.545563 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:30.546270 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.546873 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.547892 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.548093 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.548159 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.549079 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.549081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.549081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:30.549086 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.549115 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:30.549121 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.549183 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.549270 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.549307 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:30.549460 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal/instance
I20260808 07:10:30.549661 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.544581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42125 (local address 127.18.63.66:55367)
0808 07:10:30.544762 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.544798 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.545292 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.545622 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.545625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.545634 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:30.546173 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.546178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.546974 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.546975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.547794 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.547796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.548182 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.548183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.548301 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.548927 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:30.548927 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:30.548957 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:30.548982 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:30.549192 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:30.549195 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:30.549197 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.549247 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.549332 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 07:10:30.549710 18685 external_mini_cluster.cc:1428] Setting key c590aa1fc1fa67c8ad47bbcb48d9953b
I20260808 07:10:30.550274 12638 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42125
I20260808 07:10:30.550385 12638 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:30.550609 12638 heartbeater.cc:511] Master 127.18.63.126:42125 requested a full tablet report, sending...
I20260808 07:10:30.551057 12187 ts_manager.cc:194] Registered new tserver with Master: fc93cf34a491447fa3ebf4aae3fc4b15 (127.18.63.66:35187)
I20260808 07:10:30.551102 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.63.67:0
--local_ip_for_outbound_sockets=127.18.63.67
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--builtin_ntp_servers=127.18.63.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 07:10:30.551494 12187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.66:55367
W20260808 07:10:30.641604 12642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:30.641811 12642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:30.641834 12642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:30.642952 12642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:30.642990 12642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:30.643015 12642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.63.67
I20260808 07:10:30.644281 12642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.63.67
--webserver_port=0
--tserver_master_addrs=127.18.63.126:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.63.67
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:30.644483 12642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:30.644737 12642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:30.650099 12651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:30.650149 12647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.650239 12642 server_base.cc:1034] running on GCE node
W20260808 07:10:30.650329 12648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:30.650611 12642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:30.650878 12642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:30.652079 12642 hybrid_clock.cc:648] HybridClock initialized: now 1786173030652069 us; error 37 us; skew 500 ppm
I20260808 07:10:30.653533 12642 webserver.cc:460] Webserver started at http://127.18.63.67:34297/ using document root <none> and password file <none>
I20260808 07:10:30.653791 12642 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:30.653861 12642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:30.654017 12642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:30.655205 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data/instance:
uuid: "5b235ae60510440db1cb616d06975960"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "1392fda9b2a26149b22bc8c126896e20"
server_key_iv: "57113574fc20a8ff5c39b1c52f3fce10"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.655583 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance:
uuid: "5b235ae60510440db1cb616d06975960"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "1392fda9b2a26149b22bc8c126896e20"
server_key_iv: "57113574fc20a8ff5c39b1c52f3fce10"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.657191 12642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:30.658247 12660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.658439 12642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:30.658591 12642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal
uuid: "5b235ae60510440db1cb616d06975960"
format_stamp: "Formatted at 2026-08-08 07:10:30 on dist-test-slave-bw88"
server_key: "1392fda9b2a26149b22bc8c126896e20"
server_key_iv: "57113574fc20a8ff5c39b1c52f3fce10"
server_key_version: "encryptionkey@0"
I20260808 07:10:30.658674 12642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:30.697423 12642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:30.697800 12642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:30.697974 12642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:30.698258 12642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 07:10:30.698638 12642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 07:10:30.698695 12642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.698737 12642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 07:10:30.698767 12642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:30.704799 12642 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.67:44589
I20260808 07:10:30.704851 12824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.67:44589 every 8 connection(s)
I20260808 07:10:30.705199 12642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/data/info.pb
I20260808 07:10:30.708467 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 12642
I20260808 07:10:30.708588 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal/instance
I20260808 07:10:30.708806 18685 external_mini_cluster.cc:1428] Setting key 39b8d78398884b639801e2eb0ca3440a
I20260808 07:10:30.711539 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.705986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:58553 (local address 127.18.63.126:42125)
0808 07:10:30.706065 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.706067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.706330 (+ 263us) server_negotiation.cc:407] Connection header received
0808 07:10:30.706829 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.706830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.706840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.706908 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:30.707843 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.708457 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.710063 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.710279 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.710357 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.711232 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.711233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.711234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:30.711239 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.711254 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:30.711258 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.711335 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.711423 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.711456 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 07:10:30.711793 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.705968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:42125 (local address 127.18.63.67:58553)
0808 07:10:30.706191 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.706235 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.706744 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.706993 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.706996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.707005 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:10:30.707753 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.707759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.708662 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.708664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.709950 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.709953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.710380 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.710381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.710533 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.711045 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 07:10:30.711045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:10:30.711076 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 07:10:30.711100 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:10:30.711335 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:10:30.711343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:10:30.711347 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.711426 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.711556 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":118,"threads_started":1}
I20260808 07:10:30.712241 12825 heartbeater.cc:347] Connected to a master server at 127.18.63.126:42125
I20260808 07:10:30.712363 12825 heartbeater.cc:464] Registering TS with master...
I20260808 07:10:30.712563 12825 heartbeater.cc:511] Master 127.18.63.126:42125 requested a full tablet report, sending...
I20260808 07:10:30.712982 12187 ts_manager.cc:194] Registered new tserver with Master: 5b235ae60510440db1cb616d06975960 (127.18.63.67:44589)
I20260808 07:10:30.713323 12187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.63.67:58553
I20260808 07:10:30.720674 18685 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 07:10:30.724895 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.722123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.18.63.126:42125)
0808 07:10:30.722189 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.722191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.722331 (+ 140us) server_negotiation.cc:407] Connection header received
0808 07:10:30.722369 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.722370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.722381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.722430 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:30.722585 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.723039 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.723731 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.723911 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.723965 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.724639 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:30.724640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:30.724640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:30.724644 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:30.724655 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:30.724659 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:30.724702 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.724763 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.724784 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 07:10:30.726711 12187 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 07:10:30.728315 12187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 07:10:30.735083 12838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.730872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:35187 (local address 127.0.0.1:40216)
0808 07:10:30.731302 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.731313 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.731359 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.731658 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.731659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.731662 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.731715 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.731717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732615 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.732616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.734013 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.734016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.734779 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.734855 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.734868 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.734945 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.734998 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":75,"thread_start_us":84,"threads_started":1}
I20260808 07:10:30.735085 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.731107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.18.63.66:35187)
0808 07:10:30.731472 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.731476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.731487 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:10:30.731538 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.731540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.731554 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.731648 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.731773 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732475 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734113 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.734698 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734774 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.734872 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.734960 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.734999 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":96,"threads_started":1}
I20260808 07:10:30.735404 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.730918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.18.63.65:43841)
0808 07:10:30.731098 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.731103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.731113 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:30.731170 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.731171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.731183 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.731286 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.731473 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732644 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734158 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.734893 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734968 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.735053 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.735276 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.735324 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260808 07:10:30.735234 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.730653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.67:44589 (local address 127.0.0.1:34516)
0808 07:10:30.730801 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.730845 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.730913 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.731675 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.731677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.731679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.731722 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.731724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732958 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.732959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.734072 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.734074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.735011 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.735013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.735077 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.735081 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.735112 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.735150 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260808 07:10:30.735332 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.730690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43841 (local address 127.0.0.1:35898)
0808 07:10:30.730977 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.730991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.731044 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.731293 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.731296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.731298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.731374 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.731375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732710 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.732711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.734047 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.734051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.735086 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.735086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.735143 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.735145 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.735191 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.735224 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":37,"thread_start_us":61,"threads_started":1}
I20260808 07:10:30.735592 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.730780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.18.63.67:44589)
0808 07:10:30.731487 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.731491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.731501 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:30.731541 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.731543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.731570 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.731796 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.731853 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.732863 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.734160 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.734893 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.735326 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.735418 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.735445 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.735487 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":108,"threads_started":1}
I20260808 07:10:30.736335 12535 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 (DEFAULT_TABLE table=test-table [id=1dbd14e2dee44486a739f7370fe09dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.736344 12725 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 (DEFAULT_TABLE table=test-table [id=1dbd14e2dee44486a739f7370fe09dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.736348 12346 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 (DEFAULT_TABLE table=test-table [id=1dbd14e2dee44486a739f7370fe09dd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 07:10:30.736718 12535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8efdf429b9487fb67cf63e0f6400c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.736716 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8efdf429b9487fb67cf63e0f6400c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.736716 12346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8efdf429b9487fb67cf63e0f6400c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:30.739578 12842 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Bootstrap starting.
I20260808 07:10:30.740192 12844 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Bootstrap starting.
I20260808 07:10:30.740342 12842 tablet_bootstrap.cc:654] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.740423 12843 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Bootstrap starting.
I20260808 07:10:30.740739 12842 log.cc:824] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.741111 12843 tablet_bootstrap.cc:654] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.741513 12843 log.cc:824] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.741739 12842 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: No bootstrap required, opened a new log
I20260808 07:10:30.741815 12842 ts_tablet_manager.cc:1397] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 07:10:30.742410 12844 tablet_bootstrap.cc:654] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:30.742789 12844 log.cc:824] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:30.743346 12843 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: No bootstrap required, opened a new log
I20260808 07:10:30.743424 12843 ts_tablet_manager.cc:1397] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 07:10:30.744621 12842 raft_consensus.cc:348] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.744801 12842 raft_consensus.cc:374] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.744843 12842 raft_consensus.cc:729] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc93cf34a491447fa3ebf4aae3fc4b15, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.744964 12842 consensus_queue.cc:260] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.745245 12842 ts_tablet_manager.cc:1428] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 07:10:30.745265 12638 heartbeater.cc:503] Master 127.18.63.126:42125 was elected leader, sending a full tablet report...
I20260808 07:10:30.745645 12844 tablet_bootstrap.cc:492] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: No bootstrap required, opened a new log
I20260808 07:10:30.745723 12844 ts_tablet_manager.cc:1397] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 07:10:30.747722 12844 raft_consensus.cc:348] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.747844 12844 raft_consensus.cc:374] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.747865 12844 raft_consensus.cc:729] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b235ae60510440db1cb616d06975960, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.747999 12844 consensus_queue.cc:260] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.748245 12844 ts_tablet_manager.cc:1428] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 07:10:30.748333 12825 heartbeater.cc:503] Master 127.18.63.126:42125 was elected leader, sending a full tablet report...
I20260808 07:10:30.749024 12843 raft_consensus.cc:348] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.749202 12843 raft_consensus.cc:374] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:30.749257 12843 raft_consensus.cc:729] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd2fb3a1f424c5a82285b70504cebcd, State: Initialized, Role: FOLLOWER
I20260808 07:10:30.749393 12843 consensus_queue.cc:260] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.749607 12843 ts_tablet_manager.cc:1428] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260808 07:10:30.749730 12439 heartbeater.cc:503] Master 127.18.63.126:42125 was elected leader, sending a full tablet report...
W20260808 07:10:30.794363 12639 tablet.cc:2369] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:30.897490 12441 tablet.cc:2369] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 07:10:30.955821 12826 tablet.cc:2369] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 07:10:30.977150 12850 raft_consensus.cc:482] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 07:10:30.977303 12850 raft_consensus.cc:504] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.977777 12850 leader_election.cc:288] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc93cf34a491447fa3ebf4aae3fc4b15 (127.18.63.66:35187), 7bd2fb3a1f424c5a82285b70504cebcd (127.18.63.65:43841)
I20260808 07:10:30.981834 12840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.978074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:52293 (local address 127.18.63.65:43841)
0808 07:10:30.978134 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.978136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.978335 (+ 199us) server_negotiation.cc:407] Connection header received
0808 07:10:30.978362 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.978363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.978374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.978413 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.978586 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.979282 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.980842 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.981421 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.981501 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.981509 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.981738 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.981777 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:10:30.981824 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.978034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.63.67:42871 (local address 127.18.63.66:35187)
0808 07:10:30.978086 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:30.978090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:30.978102 (+ 12us) server_negotiation.cc:407] Connection header received
0808 07:10:30.978183 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:30.978189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:30.978200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:30.978245 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 07:10:30.978497 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.979282 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.980842 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.981433 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.981501 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.981509 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:30.981704 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 07:10:30.981751 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:30.981841 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.977953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.66:35187 (local address 127.18.63.67:42871)
0808 07:10:30.978043 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.978056 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.978104 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.978282 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.978283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.978285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.978378 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.978385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.979374 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.979374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.980744 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.980748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.981518 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.981519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.981614 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.981619 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.981692 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.981751 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":98}
I20260808 07:10:30.981850 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:30.977997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43841 (local address 127.18.63.67:52293)
0808 07:10:30.978216 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:30.978243 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 07:10:30.978304 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:10:30.978435 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:10:30.978436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:10:30.978438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 07:10:30.978515 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.978516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.979374 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.979375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.980744 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:10:30.980748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:10:30.981518 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:10:30.981519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:10:30.981632 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:30.981636 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 07:10:30.981727 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 07:10:30.981764 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":53,"thread_start_us":126,"threads_started":1}
I20260808 07:10:30.982288 12377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8efdf429b9487fb67cf63e0f6400c3" candidate_uuid: "5b235ae60510440db1cb616d06975960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" is_pre_election: true
I20260808 07:10:30.982287 12564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8efdf429b9487fb67cf63e0f6400c3" candidate_uuid: "5b235ae60510440db1cb616d06975960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" is_pre_election: true
I20260808 07:10:30.982451 12377 raft_consensus.cc:2454] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b235ae60510440db1cb616d06975960 in term 0.
I20260808 07:10:30.982530 12564 raft_consensus.cc:2454] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b235ae60510440db1cb616d06975960 in term 0.
I20260808 07:10:30.982712 12666 leader_election.cc:302] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b235ae60510440db1cb616d06975960, 7bd2fb3a1f424c5a82285b70504cebcd; no voters:
I20260808 07:10:30.982872 12850 raft_consensus.cc:2781] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 07:10:30.982932 12850 raft_consensus.cc:482] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 07:10:30.982959 12850 raft_consensus.cc:3037] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.983741 12850 raft_consensus.cc:504] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.983911 12850 leader_election.cc:288] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [CANDIDATE]: Term 1 election: Requested vote from peers fc93cf34a491447fa3ebf4aae3fc4b15 (127.18.63.66:35187), 7bd2fb3a1f424c5a82285b70504cebcd (127.18.63.65:43841)
I20260808 07:10:30.984054 12564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8efdf429b9487fb67cf63e0f6400c3" candidate_uuid: "5b235ae60510440db1cb616d06975960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15"
I20260808 07:10:30.984119 12377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8efdf429b9487fb67cf63e0f6400c3" candidate_uuid: "5b235ae60510440db1cb616d06975960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd2fb3a1f424c5a82285b70504cebcd"
I20260808 07:10:30.984128 12564 raft_consensus.cc:3037] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.984205 12377 raft_consensus.cc:3037] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:30.984931 12564 raft_consensus.cc:2454] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b235ae60510440db1cb616d06975960 in term 1.
I20260808 07:10:30.984959 12377 raft_consensus.cc:2454] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b235ae60510440db1cb616d06975960 in term 1.
I20260808 07:10:30.985119 12664 leader_election.cc:302] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b235ae60510440db1cb616d06975960, fc93cf34a491447fa3ebf4aae3fc4b15; no voters:
I20260808 07:10:30.985249 12850 raft_consensus.cc:2781] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:30.985457 12850 raft_consensus.cc:686] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 1 LEADER]: Becoming Leader. State: Replica: 5b235ae60510440db1cb616d06975960, State: Running, Role: LEADER
I20260808 07:10:30.985572 12850 consensus_queue.cc:237] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } }
I20260808 07:10:30.986446 12187 catalog_manager.cc:5495] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b235ae60510440db1cb616d06975960 (127.18.63.67). New cstate: current_term: 1 leader_uuid: "5b235ae60510440db1cb616d06975960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b235ae60510440db1cb616d06975960" member_type: VOTER last_known_addr { host: "127.18.63.67" port: 44589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 } health_report { overall_health: UNKNOWN } } }
I20260808 07:10:31.017614 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.015056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.18.63.67:44589)
0808 07:10:31.015126 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:31.015128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:31.015138 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:31.015192 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:31.015193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:31.015212 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:31.015297 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:10:31.015487 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.015956 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.016747 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:31.016951 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:31.017032 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:31.017222 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:10:31.017329 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:10:31.017461 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:31.017478 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 07:10:31.017498 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 07:10:31.019613 12564 raft_consensus.cc:1261] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 1 FOLLOWER]: Refusing update from remote peer 5b235ae60510440db1cb616d06975960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:31.019641 12377 raft_consensus.cc:1261] T 0f8efdf429b9487fb67cf63e0f6400c3 P 7bd2fb3a1f424c5a82285b70504cebcd [term 1 FOLLOWER]: Refusing update from remote peer 5b235ae60510440db1cb616d06975960: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:10:31.020004 12850 consensus_queue.cc:1035] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc93cf34a491447fa3ebf4aae3fc4b15" member_type: VOTER last_known_addr { host: "127.18.63.66" port: 35187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:31.020181 12854 consensus_queue.cc:1035] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd2fb3a1f424c5a82285b70504cebcd" member_type: VOTER last_known_addr { host: "127.18.63.65" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:10:31.022835 12865 mvcc.cc:205] Tried to move back new op lower bound from 7316164735053197312 to 7316164734917582848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164735053197312}]
I20260808 07:10:31.022843 12861 mvcc.cc:205] Tried to move back new op lower bound from 7316164735053197312 to 7316164734917582848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164735053197312}]
I20260808 07:10:31.023101 12866 mvcc.cc:205] Tried to move back new op lower bound from 7316164735053197312 to 7316164734917582848. Current Snapshot: MvccSnapshot[applied={T|T < 7316164735053197312}]
I20260808 07:10:31.024827 12187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:31.024917 12187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 07:10:31.025738 12187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa59b02377a14ceab9049a867e166eac: Sending DeleteTablet for 3 replicas of tablet 0f8efdf429b9487fb67cf63e0f6400c3
I20260808 07:10:31.026130 12535 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:31 UTC) from {username='slave'} at 127.0.0.1:40216
I20260808 07:10:31.026126 12725 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:31 UTC) from {username='slave'} at 127.0.0.1:34516
I20260808 07:10:31.026227 12346 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8efdf429b9487fb67cf63e0f6400c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 07:10:31 UTC) from {username='slave'} at 127.0.0.1:35898
I20260808 07:10:31.026418 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 12263
I20260808 07:10:31.039124 12882 tablet_replica.cc:330] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: stopping tablet replica
I20260808 07:10:31.039271 12882 raft_consensus.cc:2229] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 1 LEADER]: Raft consensus shutting down.
I20260808 07:10:31.039439 12882 raft_consensus.cc:2258] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:31.039656 12880 tablet_replica.cc:330] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: stopping tablet replica
I20260808 07:10:31.039846 12880 raft_consensus.cc:2229] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 07:10:31.040028 12880 raft_consensus.cc:2258] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 07:10:31.040099 12882 ts_tablet_manager.cc:1905] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:31.040601 12880 ts_tablet_manager.cc:1905] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 07:10:31.042078 12882 ts_tablet_manager.cc:1918] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:31.042198 12882 log.cc:1196] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-2/wal/wals/0f8efdf429b9487fb67cf63e0f6400c3
I20260808 07:10:31.042515 12882 ts_tablet_manager.cc:1939] T 0f8efdf429b9487fb67cf63e0f6400c3 P 5b235ae60510440db1cb616d06975960: Deleting consensus metadata
I20260808 07:10:31.042657 12880 ts_tablet_manager.cc:1918] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 07:10:31.042784 12880 log.cc:1196] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Deleting WAL directory at /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786172779312146-18685-0/minicluster-data/ts-1/wal/wals/0f8efdf429b9487fb67cf63e0f6400c3
I20260808 07:10:31.043016 12166 catalog_manager.cc:4841] TS 5b235ae60510440db1cb616d06975960 (127.18.63.67:44589): tablet 0f8efdf429b9487fb67cf63e0f6400c3 (table test-table [id=1dbd14e2dee44486a739f7370fe09dd8]) successfully deleted
I20260808 07:10:31.043119 12880 ts_tablet_manager.cc:1939] T 0f8efdf429b9487fb67cf63e0f6400c3 P fc93cf34a491447fa3ebf4aae3fc4b15: Deleting consensus metadata
I20260808 07:10:31.043506 12166 catalog_manager.cc:4841] TS fc93cf34a491447fa3ebf4aae3fc4b15 (127.18.63.66:35187): tablet 0f8efdf429b9487fb67cf63e0f6400c3 (table test-table [id=1dbd14e2dee44486a739f7370fe09dd8]) successfully deleted
W20260808 07:10:31.044976 12169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.63.65:43841 (error 108)
I20260808 07:10:31.045513 12838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:31.045374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.65:43841 (local address 127.0.0.1:35912)
0808 07:10:31.045428 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:10:31.045471 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.65:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:10:31.045647 12169 catalog_manager.cc:4584] TS 7bd2fb3a1f424c5a82285b70504cebcd (127.18.63.65:43841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f8efdf429b9487fb67cf63e0f6400c3: Network error: Client connection negotiation failed: client connection to 127.18.63.65:43841: connect: Connection refused (error 111)
I20260808 07:10:31.045836 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 12448
I20260808 07:10:31.054483 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 12642
I20260808 07:10:31.064760 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 12144
2026-08-08T07:10:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (968 ms)
[----------] 6 tests from EncryptionPolicyTest (9760 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:31 dist-test-slave-bw88 krb5kdc[12888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:31 dist-test-slave-bw88 krb5kdc[12888](info): set up 2 sockets
Aug 08 07:10:31 dist-test-slave-bw88 krb5kdc[12888](info): commencing operation
krb5kdc: starting...
W20260808 07:10:32.260708 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:32.390308 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:32.473971 12906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:32.474234 12906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:32.474272 12906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:32.475411 12906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:32.475488 12906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:32.475513 12906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:32.475534 12906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:32.475564 12906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:32.477064 12906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:32.477376 12906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:32.477618 12906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:32.482790 12921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.482826 12916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:32.482849 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:32.483174 12906 server_base.cc:1034] running on GCE node
I20260808 07:10:32.483420 12906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:32.483711 12906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:32.484875 12906 hybrid_clock.cc:648] HybridClock initialized: now 1786173032484873 us; error 44 us; skew 500 ppm
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[12888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:32.497637 12906 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:32.498116 12906 webserver.cc:460] Webserver started at http://127.0.0.1:45659/ using document root <none> and password file <none>
I20260808 07:10:32.498370 12906 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:32.498445 12906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:32.498610 12906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:32.499711 12906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "3581aba461e643af99bc64ca64a71bef"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-bw88"
server_key: "e6574da56f922242c04697e132a49006"
server_key_iv: "d1edbb73c10b5bcdd3b4f77ffbe15953"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.500099 12906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "3581aba461e643af99bc64ca64a71bef"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-bw88"
server_key: "e6574da56f922242c04697e132a49006"
server_key_iv: "d1edbb73c10b5bcdd3b4f77ffbe15953"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.501675 12906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 07:10:32.502542 12928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:32.502786 12906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:32.502907 12906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "3581aba461e643af99bc64ca64a71bef"
format_stamp: "Formatted at 2026-08-08 07:10:32 on dist-test-slave-bw88"
server_key: "e6574da56f922242c04697e132a49006"
server_key_iv: "d1edbb73c10b5bcdd3b4f77ffbe15953"
server_key_version: "encryptionkey@0"
I20260808 07:10:32.502990 12906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:32.522634 12906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:32.524523 12906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:32.524695 12906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:32.528987 12906 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38215
I20260808 07:10:32.529026 13008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38215 every 8 connection(s)
I20260808 07:10:32.529488 12906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:32.530422 13009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:32.532805 13009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef: Bootstrap starting.
I20260808 07:10:32.533512 13009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:32.533823 13009 log.cc:824] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:32.534752 13009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef: No bootstrap required, opened a new log
I20260808 07:10:32.536290 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 12906
I20260808 07:10:32.536398 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:32.536631 18685 external_mini_cluster.cc:1428] Setting key cc7d678f45b80868ea6cbdcb188eba2c
I20260808 07:10:32.536890 13009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 07:10:32.537103 13009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:32.537214 13009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3581aba461e643af99bc64ca64a71bef, State: Initialized, Role: FOLLOWER
I20260808 07:10:32.537467 13009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 07:10:32.537596 13009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:32.537633 13009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:32.537686 13009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:32.538941 13009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 07:10:32.539070 13009 leader_election.cc:302] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3581aba461e643af99bc64ca64a71bef; no voters:
I20260808 07:10:32.539247 13009 leader_election.cc:288] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:32.539306 13014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:32.539469 13014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [term 1 LEADER]: Becoming Leader. State: Replica: 3581aba461e643af99bc64ca64a71bef, State: Running, Role: LEADER
I20260808 07:10:32.539536 13009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:32.539678 13014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 07:10:32.540197 13015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3581aba461e643af99bc64ca64a71bef. Latest consensus state: current_term: 1 leader_uuid: "3581aba461e643af99bc64ca64a71bef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260808 07:10:32.540374 13015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:32.540629 13016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3581aba461e643af99bc64ca64a71bef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581aba461e643af99bc64ca64a71bef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260808 07:10:32.540773 13016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:32.540884 13023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:32.541461 13023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:32.543116 13023 catalog_manager.cc:1347] Generated new cluster ID: fd81a949ecd14aa68e1b66a9f2fee49b
I20260808 07:10:32.543169 13023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[12888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:10:32.549381 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.537286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40375 (local address 127.0.0.1:38215)
0808 07:10:32.537511 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.537521 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.537559 (+ 38us) server_negotiation.cc:407] Connection header received
0808 07:10:32.537709 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.537717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.537782 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.537960 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.538588 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.540133 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.541430 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.542043 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.542175 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.545386 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.545390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.545390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:32.545403 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.547467 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.547972 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.547974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.547975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.548092 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.548370 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.548371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.548371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.548689 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.548838 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.549057 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.549161 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":121,"threads_started":1}
I20260808 07:10:32.550287 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:32.561581 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:32.553738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53147 (local address 127.0.0.1:38215)
0808 07:10:32.553784 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:32.553786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:32.553878 (+ 92us) server_negotiation.cc:407] Connection header received
0808 07:10:32.553945 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:32.553946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:32.553957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:32.554012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:32.554764 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.555492 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.556623 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:32.556952 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:32.557006 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:32.558587 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:32.558588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:32.558589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:32.558596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:32.560195 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.560664 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.560665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.560665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.560720 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:32.560965 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:32.560965 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:32.560966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:32.561083 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:32.561197 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:32.561416 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 07:10:32.561479 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:32.569856 13023 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:32.570696 13023 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:32.574559 13023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3581aba461e643af99bc64ca64a71bef: Generated new TSK 0
I20260808 07:10:32.574759 13023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:32.577625 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 12906
W20260808 07:10:32.594535 12886 connection.cc:535] client connection to 127.0.0.1:38215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38215): Connection reset by peer (error 104)
2026-08-08T07:10:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[13050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[13050](info): set up 2 sockets
Aug 08 07:10:32 dist-test-slave-bw88 krb5kdc[13050](info): commencing operation
krb5kdc: starting...
W20260808 07:10:33.780086 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:33 dist-test-slave-bw88 krb5kdc[13050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:33.908885 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:33.993867 13072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:33.994083 13072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:33.994107 13072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:33.995260 13072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:33.995303 13072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:33.995318 13072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:33.995332 13072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:33.995343 13072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:33.996837 13072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41965
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:33.997104 13072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:33.997331 13072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:34.002660 13085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:34.002669 13078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:34.002749 13080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:34.002815 13072 server_base.cc:1034] running on GCE node
I20260808 07:10:34.003137 13072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:34.003466 13072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:34.004654 13072 hybrid_clock.cc:648] HybridClock initialized: now 1786173034004629 us; error 45 us; skew 500 ppm
Aug 08 07:10:34 dist-test-slave-bw88 krb5kdc[13050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:34.007331 13072 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:34.007910 13072 webserver.cc:460] Webserver started at http://127.0.0.1:37721/ using document root <none> and password file <none>
I20260808 07:10:34.008172 13072 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:34.008250 13072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:34.008415 13072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:34.009720 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "19efa4dbe4e74651afa2d8f43d5c0816"
format_stamp: "Formatted at 2026-08-08 07:10:34 on dist-test-slave-bw88"
server_key: "dbd9d1c28bfa2f39f74100884b064e21"
server_key_iv: "7a48bfde57d6e4b1cb798a9735efbc57"
server_key_version: "encryptionkey@0"
I20260808 07:10:34.010155 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "19efa4dbe4e74651afa2d8f43d5c0816"
format_stamp: "Formatted at 2026-08-08 07:10:34 on dist-test-slave-bw88"
server_key: "dbd9d1c28bfa2f39f74100884b064e21"
server_key_iv: "7a48bfde57d6e4b1cb798a9735efbc57"
server_key_version: "encryptionkey@0"
I20260808 07:10:34.011752 13072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:10:34.012599 13094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:34.012790 13072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:34.012913 13072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "19efa4dbe4e74651afa2d8f43d5c0816"
format_stamp: "Formatted at 2026-08-08 07:10:34 on dist-test-slave-bw88"
server_key: "dbd9d1c28bfa2f39f74100884b064e21"
server_key_iv: "7a48bfde57d6e4b1cb798a9735efbc57"
server_key_version: "encryptionkey@0"
I20260808 07:10:34.013002 13072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:34.020790 13072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:34.022738 13072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:34.022912 13072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:34.027549 13072 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41965
I20260808 07:10:34.027590 13169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41965 every 8 connection(s)
I20260808 07:10:34.027942 13072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:34.028841 13171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:34.031169 13171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816: Bootstrap starting.
I20260808 07:10:34.031759 13171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:34.032048 13171 log.cc:824] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:34.032835 13171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816: No bootstrap required, opened a new log
I20260808 07:10:34.034132 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13072
I20260808 07:10:34.034289 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:34.034505 18685 external_mini_cluster.cc:1428] Setting key f1f3fbe8a1d00513dd6b2aa2612c640b
I20260808 07:10:34.034687 13171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260808 07:10:34.034874 13171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:34.034960 13171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19efa4dbe4e74651afa2d8f43d5c0816, State: Initialized, Role: FOLLOWER
I20260808 07:10:34.035198 13171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260808 07:10:34.035307 13171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:34.035342 13171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:34.035382 13171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:34.036125 13171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260808 07:10:34.036234 13171 leader_election.cc:302] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19efa4dbe4e74651afa2d8f43d5c0816; no voters:
I20260808 07:10:34.036383 13171 leader_election.cc:288] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:34.036461 13177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:34.036623 13171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:34.036646 13177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [term 1 LEADER]: Becoming Leader. State: Replica: 19efa4dbe4e74651afa2d8f43d5c0816, State: Running, Role: LEADER
I20260808 07:10:34.036769 13177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } }
I20260808 07:10:34.037146 13178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } } }
I20260808 07:10:34.037220 13180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19efa4dbe4e74651afa2d8f43d5c0816. Latest consensus state: current_term: 1 leader_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19efa4dbe4e74651afa2d8f43d5c0816" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41965 } } }
I20260808 07:10:34.037369 13180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:34.037282 13178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:34.037637 13195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:34.038340 13195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:34.039968 13195 catalog_manager.cc:1347] Generated new cluster ID: 5c509a5acfd84a1c9a284d7c8ef45762
I20260808 07:10:34.040081 13195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:34.040727 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:34.035088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53065 (local address 127.0.0.1:41965)
0808 07:10:34.035282 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:34.035291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:34.035327 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:34.035442 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:34.035449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:34.035503 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:34.035659 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:34.036037 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:34.037439 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:34.038720 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:34.039278 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:34.039385 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:34.040109 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:34.040113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:34.040113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:34.040304 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:34.040333 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:34.040344 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:34.040438 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:34.040524 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 07:10:34.040551 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260808 07:10:34.041762 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:34.048261 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:34.043111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34299 (local address 127.0.0.1:41965)
0808 07:10:34.043148 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:34.043150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:34.045515 (+ 2365us) server_negotiation.cc:407] Connection header received
0808 07:10:34.045610 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:34.045611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:34.045622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:34.045678 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:34.045820 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:34.046309 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:34.047033 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:34.047257 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:34.047343 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:34.047993 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:34.047995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:34.047995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:34.047999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:34.048012 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:34.048016 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:34.048084 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:34.048149 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 07:10:34.048167 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:34.075758 13195 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:34.076311 13195 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:34.088871 13195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19efa4dbe4e74651afa2d8f43d5c0816: Generated new TSK 0
I20260808 07:10:34.089072 13195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:34.092365 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13072
W20260808 07:10:34.102427 13048 connection.cc:535] client connection to 127.0.0.1:41965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41965): Connection reset by peer (error 104)
2026-08-08T07:10:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:34 dist-test-slave-bw88 krb5kdc[13219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:34 dist-test-slave-bw88 krb5kdc[13219](info): set up 2 sockets
Aug 08 07:10:34 dist-test-slave-bw88 krb5kdc[13219](info): commencing operation
krb5kdc: starting...
W20260808 07:10:35.377096 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:35 dist-test-slave-bw88 krb5kdc[13219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:35.504796 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:35.588833 13238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:35.589098 13238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:35.589140 13238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:35.590303 13238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:35.590382 13238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:35.590408 13238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:35.590430 13238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:35.590461 13238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:35.591948 13238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:35.592247 13238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:35.592495 13238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:35.597608 13249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:35.597621 13247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.597867 13238 server_base.cc:1034] running on GCE node
W20260808 07:10:35.597640 13244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:35.598351 13238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:35.598701 13238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:35.599856 13238 hybrid_clock.cc:648] HybridClock initialized: now 1786173035599833 us; error 41 us; skew 500 ppm
Aug 08 07:10:35 dist-test-slave-bw88 krb5kdc[13219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:35.602573 13238 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:35.603122 13238 webserver.cc:460] Webserver started at http://127.0.0.1:43045/ using document root <none> and password file <none>
I20260808 07:10:35.603399 13238 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:35.603477 13238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:35.603637 13238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:35.604779 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "b58fc31ff64042bfa2f3fe8e9e087a40"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-bw88"
server_key: "720f386c0994c97d82f4180df319dc01"
server_key_iv: "b62e10376ee3353da9c45fbbd9cb2e2b"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.605175 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "b58fc31ff64042bfa2f3fe8e9e087a40"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-bw88"
server_key: "720f386c0994c97d82f4180df319dc01"
server_key_iv: "b62e10376ee3353da9c45fbbd9cb2e2b"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.606662 13238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:35.607434 13259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.607613 13238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:35.607728 13238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "b58fc31ff64042bfa2f3fe8e9e087a40"
format_stamp: "Formatted at 2026-08-08 07:10:35 on dist-test-slave-bw88"
server_key: "720f386c0994c97d82f4180df319dc01"
server_key_iv: "b62e10376ee3353da9c45fbbd9cb2e2b"
server_key_version: "encryptionkey@0"
I20260808 07:10:35.607807 13238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:35.611645 13238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:35.614895 13238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:35.615064 13238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:35.620496 13238 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44399
I20260808 07:10:35.620534 13330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44399 every 8 connection(s)
I20260808 07:10:35.620915 13238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:35.622085 13332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:35.624428 13332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40: Bootstrap starting.
I20260808 07:10:35.625030 13332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:35.625366 13332 log.cc:824] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:35.626120 13332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40: No bootstrap required, opened a new log
I20260808 07:10:35.627753 13332 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } }
I20260808 07:10:35.627911 13332 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:35.627964 13332 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58fc31ff64042bfa2f3fe8e9e087a40, State: Initialized, Role: FOLLOWER
I20260808 07:10:35.628129 13332 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } }
I20260808 07:10:35.628230 13332 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:35.628273 13332 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:35.628324 13332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:35.628871 13332 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } }
I20260808 07:10:35.628993 13332 leader_election.cc:302] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58fc31ff64042bfa2f3fe8e9e087a40; no voters:
I20260808 07:10:35.629168 13332 leader_election.cc:288] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:35.629221 13336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:35.629444 13336 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [term 1 LEADER]: Becoming Leader. State: Replica: b58fc31ff64042bfa2f3fe8e9e087a40, State: Running, Role: LEADER
I20260808 07:10:35.629458 13332 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:35.629616 13336 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } }
I20260808 07:10:35.630198 13337 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } } }
I20260808 07:10:35.630301 13337 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:35.630528 13351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:35.630654 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13238
I20260808 07:10:35.630789 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:35.630791 13338 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58fc31ff64042bfa2f3fe8e9e087a40. Latest consensus state: current_term: 1 leader_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58fc31ff64042bfa2f3fe8e9e087a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44399 } } }
I20260808 07:10:35.630873 13338 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:35.631013 18685 external_mini_cluster.cc:1428] Setting key 5825124623bee357a8de3227d933f62b
I20260808 07:10:35.631165 13351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:35.632732 13351 catalog_manager.cc:1347] Generated new cluster ID: 9c3b532db4e14e9caec7598cf6ec0815
I20260808 07:10:35.632787 13351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:35.634972 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.631526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40363 (local address 127.0.0.1:44399)
0808 07:10:35.633538 (+ 2012us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:35.633547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:35.633580 (+ 33us) server_negotiation.cc:407] Connection header received
0808 07:10:35.633690 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:35.633696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:35.633724 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:35.633860 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:35.634547 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:35.634550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:35.634550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:35.634553 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:35.634587 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:35.634600 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:35.634656 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:35.634751 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:10:35.634790 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1959,"thread_start_us":96,"threads_started":1}
I20260808 07:10:35.638618 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:35.641157 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:35.640061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60461 (local address 127.0.0.1:44399)
0808 07:10:35.640102 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:35.640104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:35.640243 (+ 139us) server_negotiation.cc:407] Connection header received
0808 07:10:35.640299 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:35.640300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:35.640306 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:35.640359 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:35.640940 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:35.640941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:35.640941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:35.640945 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:35.640960 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:35.640964 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:35.641024 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:35.641087 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:10:35.641098 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:10:35.654861 13351 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:35.655035 13351 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:35.661216 13351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58fc31ff64042bfa2f3fe8e9e087a40: Generated new TSK 0
I20260808 07:10:35.661415 13351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:35.666651 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13238
2026-08-08T07:10:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:35 dist-test-slave-bw88 krb5kdc[13372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:35 dist-test-slave-bw88 krb5kdc[13372](info): set up 2 sockets
Aug 08 07:10:35 dist-test-slave-bw88 krb5kdc[13372](info): commencing operation
krb5kdc: starting...
W20260808 07:10:36.853183 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:36 dist-test-slave-bw88 krb5kdc[13372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:36.985317 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:37.069149 13387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:37.069411 13387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:37.069476 13387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:37.070649 13387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:37.070693 13387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:37.070708 13387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:37.070721 13387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:37.070734 13387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:37.072216 13387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:37.072468 13387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:37.072717 13387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:37.077975 13395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:37.078007 13399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.078084 13387 server_base.cc:1034] running on GCE node
W20260808 07:10:37.077989 13394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:37.078392 13387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:37.078737 13387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:37.079916 13387 hybrid_clock.cc:648] HybridClock initialized: now 1786173037079901 us; error 49 us; skew 500 ppm
Aug 08 07:10:37 dist-test-slave-bw88 krb5kdc[13372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173037, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:37.082590 13387 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:37.082973 13387 webserver.cc:460] Webserver started at http://127.0.0.1:34971/ using document root <none> and password file <none>
I20260808 07:10:37.083189 13387 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:37.083230 13387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:37.083317 13387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:37.084329 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "69b380e139154fecb68f44b7b82fd5da"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-bw88"
server_key: "707f614aa8c02b9a0bace4792fbce557"
server_key_iv: "27f38c9ece0cc504bc6a58d8ccf2e1ca"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.084640 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "69b380e139154fecb68f44b7b82fd5da"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-bw88"
server_key: "707f614aa8c02b9a0bace4792fbce557"
server_key_iv: "27f38c9ece0cc504bc6a58d8ccf2e1ca"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.086215 13387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:37.087208 13410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:37.087442 13387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:37.087543 13387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "69b380e139154fecb68f44b7b82fd5da"
format_stamp: "Formatted at 2026-08-08 07:10:37 on dist-test-slave-bw88"
server_key: "707f614aa8c02b9a0bace4792fbce557"
server_key_iv: "27f38c9ece0cc504bc6a58d8ccf2e1ca"
server_key_version: "encryptionkey@0"
I20260808 07:10:37.087620 13387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:37.118407 13387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:37.120378 13387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:37.120500 13387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:37.124959 13387 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46695
I20260808 07:10:37.125020 13488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46695 every 8 connection(s)
I20260808 07:10:37.125412 13387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:37.126338 13489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:37.128660 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: Bootstrap starting.
I20260808 07:10:37.129274 13489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:37.129599 13489 log.cc:824] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:37.130363 13489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: No bootstrap required, opened a new log
I20260808 07:10:37.130995 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13387
I20260808 07:10:37.131136 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:37.131367 18685 external_mini_cluster.cc:1428] Setting key 5a554b6082ea01b02186ce530596cf7d
I20260808 07:10:37.132395 13489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } }
I20260808 07:10:37.132608 13489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:37.132671 13489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b380e139154fecb68f44b7b82fd5da, State: Initialized, Role: FOLLOWER
I20260808 07:10:37.132824 13489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } }
I20260808 07:10:37.132921 13489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:37.132964 13489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:37.133025 13489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:37.134047 13489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } }
I20260808 07:10:37.134186 13489 leader_election.cc:302] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b380e139154fecb68f44b7b82fd5da; no voters:
I20260808 07:10:37.134348 13489 leader_election.cc:288] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:37.134392 13498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:37.134543 13498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [term 1 LEADER]: Becoming Leader. State: Replica: 69b380e139154fecb68f44b7b82fd5da, State: Running, Role: LEADER
I20260808 07:10:37.134631 13489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:37.134690 13498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } }
I20260808 07:10:37.135143 13498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b380e139154fecb68f44b7b82fd5da. Latest consensus state: current_term: 1 leader_uuid: "69b380e139154fecb68f44b7b82fd5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } } }
I20260808 07:10:37.135152 13499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69b380e139154fecb68f44b7b82fd5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b380e139154fecb68f44b7b82fd5da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46695 } } }
I20260808 07:10:37.135300 13498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:37.135399 13499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da [sys.catalog]: This master's current role is: LEADER
W20260808 07:10:37.136668 13520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:37.136794 13520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:37.136895 13511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:37.137095 13511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:37.138710 13511 catalog_manager.cc:1347] Generated new cluster ID: 24b5b410349b4ad9828a3b8e78647258
I20260808 07:10:37.138763 13511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:37 dist-test-slave-bw88 krb5kdc[13372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:10:37.143968 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:37.131963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56263 (local address 127.0.0.1:46695)
0808 07:10:37.132154 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:37.132163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:37.132189 (+ 26us) server_negotiation.cc:407] Connection header received
0808 07:10:37.132352 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:37.132357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:37.132400 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:37.132512 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:37.133530 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.134684 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.135847 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.136604 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.136807 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:37.139750 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:37.139754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:37.139755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:37.139767 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:37.141789 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.142185 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.142188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.142189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.142319 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.142513 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.142514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.142514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.142801 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:37.143532 (+ 731us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:37.143607 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 07:10:37.143744 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260808 07:10:37.145009 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:37.153667 13496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:37.146512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51523 (local address 127.0.0.1:46695)
0808 07:10:37.146547 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:37.146549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:37.146656 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:10:37.146736 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:37.146737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:37.146746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:37.146772 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:37.147411 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.148159 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.149130 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:37.149483 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:37.149523 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:37.150979 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:37.150980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:37.150981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:37.150989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:37.152522 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.152881 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.152882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.152882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.152931 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:37.153113 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:37.153114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:37.153114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:37.153225 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:37.153288 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:37.153496 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 07:10:37.153572 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:10:37.167348 13511 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:37.167917 13511 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:37.171160 13511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69b380e139154fecb68f44b7b82fd5da: Generated new TSK 0
I20260808 07:10:37.171353 13511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:37.179167 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13387
2026-08-08T07:10:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:37 dist-test-slave-bw88 krb5kdc[13533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:37 dist-test-slave-bw88 krb5kdc[13533](info): set up 2 sockets
Aug 08 07:10:37 dist-test-slave-bw88 krb5kdc[13533](info): commencing operation
krb5kdc: starting...
W20260808 07:10:38.378756 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:38 dist-test-slave-bw88 krb5kdc[13533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:38.512212 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:38.596904 13551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:38.597165 13551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:38.597208 13551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:38.598415 13551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:38.598493 13551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:38.598519 13551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:38.598559 13551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:38.598585 13551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:38.600114 13551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:38.600394 13551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:38.600649 13551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:38.605836 13560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.605872 13556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:38.605842 13558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:38.606170 13551 server_base.cc:1034] running on GCE node
I20260808 07:10:38.606487 13551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:38.606757 13551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:38.607931 13551 hybrid_clock.cc:648] HybridClock initialized: now 1786173038607918 us; error 43 us; skew 500 ppm
Aug 08 07:10:38 dist-test-slave-bw88 krb5kdc[13533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:38.616781 13551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:38.617292 13551 webserver.cc:460] Webserver started at http://127.0.0.1:34057/ using document root <none> and password file <none>
I20260808 07:10:38.617596 13551 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:38.617677 13551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:38.617843 13551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:38.618958 13551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "4296345d15b6470796246003193bc684"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-bw88"
server_key: "b1d511331edb3836b948f53dc1ec701a"
server_key_iv: "dc4bec7c39eccefa67e6ec5bad2f1b4c"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.619349 13551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "4296345d15b6470796246003193bc684"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-bw88"
server_key: "b1d511331edb3836b948f53dc1ec701a"
server_key_iv: "dc4bec7c39eccefa67e6ec5bad2f1b4c"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.620729 13551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:38.621697 13572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:38.621935 13551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:38.622063 13551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "4296345d15b6470796246003193bc684"
format_stamp: "Formatted at 2026-08-08 07:10:38 on dist-test-slave-bw88"
server_key: "b1d511331edb3836b948f53dc1ec701a"
server_key_iv: "dc4bec7c39eccefa67e6ec5bad2f1b4c"
server_key_version: "encryptionkey@0"
I20260808 07:10:38.622143 13551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:38.636253 13551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:38.638723 13551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:38.638903 13551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:38.643098 13551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35455
I20260808 07:10:38.643138 13640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35455 every 8 connection(s)
I20260808 07:10:38.643550 13551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:38.644520 13642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:38.646953 13642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684: Bootstrap starting.
I20260808 07:10:38.647516 13642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:38.647770 13642 log.cc:824] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:38.648260 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13551
I20260808 07:10:38.648391 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:38.648604 18685 external_mini_cluster.cc:1428] Setting key 9bff3b1934f1121c9362df17ebc65a30
I20260808 07:10:38.648598 13642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684: No bootstrap required, opened a new log
I20260808 07:10:38.651310 13642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260808 07:10:38.651500 13642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:38.651543 13642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4296345d15b6470796246003193bc684, State: Initialized, Role: FOLLOWER
I20260808 07:10:38.651674 13642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260808 07:10:38.651754 13642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:38.651793 13642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:38.651841 13642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:38.652540 13642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260808 07:10:38.652668 13642 leader_election.cc:302] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4296345d15b6470796246003193bc684; no voters:
I20260808 07:10:38.652885 13642 leader_election.cc:288] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:38.652908 13647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:38.653041 13647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [term 1 LEADER]: Becoming Leader. State: Replica: 4296345d15b6470796246003193bc684, State: Running, Role: LEADER
I20260808 07:10:38.653177 13642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:38.653195 13647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } }
I20260808 07:10:38.653571 13647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4296345d15b6470796246003193bc684. Latest consensus state: current_term: 1 leader_uuid: "4296345d15b6470796246003193bc684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } } }
I20260808 07:10:38.653677 13647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.653584 13648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4296345d15b6470796246003193bc684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4296345d15b6470796246003193bc684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35455 } } }
I20260808 07:10:38.653790 13648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:38.653959 13657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:38.654469 13657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:38.655818 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.649144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40221 (local address 127.0.0.1:35455)
0808 07:10:38.649376 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.649386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.649422 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:38.649991 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.649999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.650048 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.650169 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.650439 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.651340 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.652490 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.653065 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.653518 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.655113 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.655117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.655118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:38.655294 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.655330 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:38.655341 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.655423 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.655513 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.655583 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260808 07:10:38.656172 13657 catalog_manager.cc:1347] Generated new cluster ID: 4b44b5c1720e41f19a4d6db1fa2ffd6b
I20260808 07:10:38.656220 13657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:38.662055 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:38.667076 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:38.663645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46517 (local address 127.0.0.1:35455)
0808 07:10:38.663687 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:38.663689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:38.663699 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:10:38.663723 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:38.663723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:38.663733 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:38.663796 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:38.663926 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.664689 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.665712 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:38.666032 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:38.666115 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:38.666815 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:38.666816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:38.666816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:38.666821 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:38.666838 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:38.666842 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:38.666902 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:38.666959 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 07:10:38.666981 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:38.683630 13657 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:38.684211 13657 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:38.688267 13657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4296345d15b6470796246003193bc684: Generated new TSK 0
I20260808 07:10:38.688465 13657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:38.692219 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13551
2026-08-08T07:10:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:38 dist-test-slave-bw88 krb5kdc[13684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:38 dist-test-slave-bw88 krb5kdc[13684](info): set up 2 sockets
Aug 08 07:10:38 dist-test-slave-bw88 krb5kdc[13684](info): commencing operation
krb5kdc: starting...
W20260808 07:10:39.972263 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:40 dist-test-slave-bw88 krb5kdc[13684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:40.100770 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37183
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:40.184885 13702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:40.185091 13702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:40.185114 13702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:40.186312 13702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:40.186357 13702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:40.186371 13702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:40.186383 13702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:40.186395 13702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:40.187877 13702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:40.188138 13702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:40.188383 13702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:40.193573 13711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:40.193593 13702 server_base.cc:1034] running on GCE node
W20260808 07:10:40.193547 13709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:40.193524 13708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:40.194052 13702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:40.194340 13702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:40.195533 13702 hybrid_clock.cc:648] HybridClock initialized: now 1786173040195526 us; error 37 us; skew 500 ppm
Aug 08 07:10:40 dist-test-slave-bw88 krb5kdc[13684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:40.198364 13702 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:40.198832 13702 webserver.cc:460] Webserver started at http://127.0.0.1:33183/ using document root <none> and password file <none>
I20260808 07:10:40.199064 13702 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:40.199132 13702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:40.199283 13702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:40.200368 13702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "1e1000c66c994e42bd6fa05ea6dbd976"
format_stamp: "Formatted at 2026-08-08 07:10:40 on dist-test-slave-bw88"
server_key: "df2c95d64a2a8b9ca22d53b4245569e9"
server_key_iv: "75700a31a49890066fddcb02d4086384"
server_key_version: "encryptionkey@0"
I20260808 07:10:40.200788 13702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "1e1000c66c994e42bd6fa05ea6dbd976"
format_stamp: "Formatted at 2026-08-08 07:10:40 on dist-test-slave-bw88"
server_key: "df2c95d64a2a8b9ca22d53b4245569e9"
server_key_iv: "75700a31a49890066fddcb02d4086384"
server_key_version: "encryptionkey@0"
I20260808 07:10:40.202245 13702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:40.203016 13724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:40.203194 13702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:10:40.203310 13702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "1e1000c66c994e42bd6fa05ea6dbd976"
format_stamp: "Formatted at 2026-08-08 07:10:40 on dist-test-slave-bw88"
server_key: "df2c95d64a2a8b9ca22d53b4245569e9"
server_key_iv: "75700a31a49890066fddcb02d4086384"
server_key_version: "encryptionkey@0"
I20260808 07:10:40.203385 13702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:40.206863 13702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:40.208699 13702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:40.208850 13702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:40.212809 13702 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37183
I20260808 07:10:40.212852 13790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37183 every 8 connection(s)
I20260808 07:10:40.213165 13702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:40.214227 13792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:40.216284 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13702
I20260808 07:10:40.216434 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:40.216771 18685 external_mini_cluster.cc:1428] Setting key f506bffc6000a1b68807799e0e7f43c3
I20260808 07:10:40.216820 13792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976: Bootstrap starting.
I20260808 07:10:40.217573 13792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:40.217901 13792 log.cc:824] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:40.218726 13792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976: No bootstrap required, opened a new log
I20260808 07:10:40.219415 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.217299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53477 (local address 127.0.0.1:37183)
0808 07:10:40.217489 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:40.217496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:40.217545 (+ 49us) server_negotiation.cc:407] Connection header received
0808 07:10:40.218084 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:40.218092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:40.218126 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:40.218270 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:40.218986 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:40.218989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:40.218990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:40.218994 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:40.219043 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:40.219054 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:40.219142 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:40.219209 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 07:10:40.219249 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260808 07:10:40.220356 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:40.220983 13792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } }
I20260808 07:10:40.221134 13792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:40.221244 13792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e1000c66c994e42bd6fa05ea6dbd976, State: Initialized, Role: FOLLOWER
I20260808 07:10:40.221422 13792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } }
I20260808 07:10:40.221527 13792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:40.221635 13792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:40.221705 13792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:40.222288 13792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } }
I20260808 07:10:40.222406 13792 leader_election.cc:302] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e1000c66c994e42bd6fa05ea6dbd976; no voters:
I20260808 07:10:40.222600 13792 leader_election.cc:288] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:40.222615 13808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:40.222759 13808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [term 1 LEADER]: Becoming Leader. State: Replica: 1e1000c66c994e42bd6fa05ea6dbd976, State: Running, Role: LEADER
I20260808 07:10:40.222865 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:40.221771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52015 (local address 127.0.0.1:37183)
0808 07:10:40.221811 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:40.221813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:40.221891 (+ 78us) server_negotiation.cc:407] Connection header received
0808 07:10:40.221964 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:40.221964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:40.221969 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:40.222032 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:40.222633 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:40.222634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:40.222635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:40.222639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:40.222652 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:40.222655 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:40.222706 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:40.222801 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:10:40.222815 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:40.222883 13792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:40.222900 13808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } }
I20260808 07:10:40.223286 13811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } } }
I20260808 07:10:40.223388 13811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:40.223592 13810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e1000c66c994e42bd6fa05ea6dbd976. Latest consensus state: current_term: 1 leader_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1000c66c994e42bd6fa05ea6dbd976" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37183 } } }
I20260808 07:10:40.223693 13810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:40.223949 13820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:40.224488 13820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:40.226037 13820 catalog_manager.cc:1347] Generated new cluster ID: 43ae18a8455e46dc89ada8b66f0da792
I20260808 07:10:40.226089 13820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:40.251994 13820 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:40.252189 13820 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:40.260689 13820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e1000c66c994e42bd6fa05ea6dbd976: Generated new TSK 0
I20260808 07:10:40.260882 13820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:40.268563 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13702
2026-08-08T07:10:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:40 dist-test-slave-bw88 krb5kdc[13840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:40 dist-test-slave-bw88 krb5kdc[13840](info): set up 2 sockets
Aug 08 07:10:40 dist-test-slave-bw88 krb5kdc[13840](info): commencing operation
krb5kdc: starting...
W20260808 07:10:41.462999 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:41.590184 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:41.673601 13862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:41.673799 13862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:41.673821 13862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:41.674949 13862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:41.674993 13862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:41.675007 13862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:41.675020 13862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:41.675032 13862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:41.676517 13862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:41.676784 13862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:41.677011 13862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:41.682121 13871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:41.682142 13874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:41.682148 13870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:41.682492 13862 server_base.cc:1034] running on GCE node
I20260808 07:10:41.682757 13862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:41.683072 13862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:41.684239 13862 hybrid_clock.cc:648] HybridClock initialized: now 1786173041684215 us; error 38 us; skew 500 ppm
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:41.686772 13862 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:41.687254 13862 webserver.cc:460] Webserver started at http://127.0.0.1:44861/ using document root <none> and password file <none>
I20260808 07:10:41.687497 13862 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:41.687569 13862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:41.687726 13862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:41.688766 13862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "69b45029b06b4698b569138fa0a2f96f"
format_stamp: "Formatted at 2026-08-08 07:10:41 on dist-test-slave-bw88"
server_key: "0d782e74548173e786ee49e5d74976ab"
server_key_iv: "c34e864e3d443a85c535fa89f759349a"
server_key_version: "encryptionkey@0"
I20260808 07:10:41.689131 13862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "69b45029b06b4698b569138fa0a2f96f"
format_stamp: "Formatted at 2026-08-08 07:10:41 on dist-test-slave-bw88"
server_key: "0d782e74548173e786ee49e5d74976ab"
server_key_iv: "c34e864e3d443a85c535fa89f759349a"
server_key_version: "encryptionkey@0"
I20260808 07:10:41.690526 13862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:41.691341 13886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:41.691550 13862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:41.691658 13862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "69b45029b06b4698b569138fa0a2f96f"
format_stamp: "Formatted at 2026-08-08 07:10:41 on dist-test-slave-bw88"
server_key: "0d782e74548173e786ee49e5d74976ab"
server_key_iv: "c34e864e3d443a85c535fa89f759349a"
server_key_version: "encryptionkey@0"
I20260808 07:10:41.691744 13862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:41.722944 13862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:41.725461 13862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:41.725643 13862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:41.729821 13862 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45449
I20260808 07:10:41.729863 13953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45449 every 8 connection(s)
I20260808 07:10:41.730268 13862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:41.731184 13956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:41.733734 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: Bootstrap starting.
I20260808 07:10:41.734333 13956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:41.734640 13956 log.cc:824] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:41.735540 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: No bootstrap required, opened a new log
I20260808 07:10:41.735776 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 13862
I20260808 07:10:41.735904 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:41.736105 18685 external_mini_cluster.cc:1428] Setting key 2752045e7eab59cdacc463cffd635c81
I20260808 07:10:41.737830 13956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260808 07:10:41.738019 13956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:41.738113 13956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b45029b06b4698b569138fa0a2f96f, State: Initialized, Role: FOLLOWER
I20260808 07:10:41.738265 13956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260808 07:10:41.738360 13956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:41.738396 13956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:41.738452 13956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:41.739445 13956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260808 07:10:41.739552 13956 leader_election.cc:302] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69b45029b06b4698b569138fa0a2f96f; no voters:
I20260808 07:10:41.739759 13956 leader_election.cc:288] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:41.739809 13964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:41.739965 13964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [term 1 LEADER]: Becoming Leader. State: Replica: 69b45029b06b4698b569138fa0a2f96f, State: Running, Role: LEADER
I20260808 07:10:41.740026 13956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:41.740173 13964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260808 07:10:41.740877 13966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b45029b06b4698b569138fa0a2f96f. Latest consensus state: current_term: 1 leader_uuid: "69b45029b06b4698b569138fa0a2f96f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } } }
I20260808 07:10:41.740962 13966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:41.741047 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69b45029b06b4698b569138fa0a2f96f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69b45029b06b4698b569138fa0a2f96f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } } }
I20260808 07:10:41.741122 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f [sys.catalog]: This master's current role is: LEADER
W20260808 07:10:41.742058 13982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:41.742658 13982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:41.742738 13975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:41.742913 13975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:10:41.744366 13975 catalog_manager.cc:1347] Generated new cluster ID: 0aba8cfb4bcc4d43bba980f00973b381
I20260808 07:10:41.744414 13975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:41.747833 13975 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:41.748409 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:41.736699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:35047 (local address 127.0.0.1:45449)
0808 07:10:41.736887 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:41.736896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:41.736932 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:41.737809 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:41.737816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:41.737868 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:41.737982 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:41.738821 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:41.739896 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:41.741102 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:41.741824 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:41.741931 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:41.744658 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:41.744663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:41.744663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:41.744677 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:41.746388 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:41.746847 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:41.746852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:41.746853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:41.746958 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:41.747230 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:41.747231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:41.747231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:41.747453 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:41.747616 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:41.748182 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 07:10:41.748267 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260808 07:10:41.748483 13975 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:41.749495 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:41.757318 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:41.750909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:57631 (local address 127.0.0.1:45449)
0808 07:10:41.750955 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:41.750958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:41.751144 (+ 186us) server_negotiation.cc:407] Connection header received
0808 07:10:41.751187 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:41.751188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:41.751199 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:41.751254 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:41.751781 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:41.752308 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:41.753062 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:41.753294 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:41.753407 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:41.754630 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:41.754632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:41.754633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:41.754639 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:41.756094 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:41.756481 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:41.756482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:41.756483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:41.756540 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:41.756786 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:41.756787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:41.756787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:41.756903 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:41.757016 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:41.757136 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 07:10:41.757202 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 07:10:41.758160 13975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69b45029b06b4698b569138fa0a2f96f: Generated new TSK 0
I20260808 07:10:41.758364 13975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:41.764340 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 13862
W20260808 07:10:41.775158 13837 connection.cc:535] client connection to 127.0.0.1:45449 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45449): Connection reset by peer (error 104)
2026-08-08T07:10:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13998](info): set up 2 sockets
Aug 08 07:10:41 dist-test-slave-bw88 krb5kdc[13998](info): commencing operation
krb5kdc: starting...
W20260808 07:10:43.188422 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:43 dist-test-slave-bw88 krb5kdc[13998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:43.320853 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:43.405988 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:43.406235 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:43.406276 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:43.407430 14015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:43.407514 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:43.407542 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:43.407574 14015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:43.407598 14015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:43.409122 14015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:43.409443 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:43.409690 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:43.414645 14023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:43.414652 14021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:43.414654 14025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:43.414904 14015 server_base.cc:1034] running on GCE node
I20260808 07:10:43.415150 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:43.415442 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:43.416639 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786173043416605 us; error 49 us; skew 500 ppm
Aug 08 07:10:43 dist-test-slave-bw88 krb5kdc[13998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:43.419469 14015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:43.419869 14015 webserver.cc:460] Webserver started at http://127.0.0.1:44441/ using document root <none> and password file <none>
I20260808 07:10:43.420097 14015 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:43.420142 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:43.420229 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:43.421267 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "11f8831d396b4e66b2f3622ced843abf"
format_stamp: "Formatted at 2026-08-08 07:10:43 on dist-test-slave-bw88"
server_key: "904af7ee1bae2f7a07d2c92eaaefc305"
server_key_iv: "234a25eb0e97b39f1a8cb7b7474d6b7d"
server_key_version: "encryptionkey@0"
I20260808 07:10:43.421658 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "11f8831d396b4e66b2f3622ced843abf"
format_stamp: "Formatted at 2026-08-08 07:10:43 on dist-test-slave-bw88"
server_key: "904af7ee1bae2f7a07d2c92eaaefc305"
server_key_iv: "234a25eb0e97b39f1a8cb7b7474d6b7d"
server_key_version: "encryptionkey@0"
I20260808 07:10:43.422933 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:43.423604 14037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:43.423812 14015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:43.423899 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "11f8831d396b4e66b2f3622ced843abf"
format_stamp: "Formatted at 2026-08-08 07:10:43 on dist-test-slave-bw88"
server_key: "904af7ee1bae2f7a07d2c92eaaefc305"
server_key_iv: "234a25eb0e97b39f1a8cb7b7474d6b7d"
server_key_version: "encryptionkey@0"
I20260808 07:10:43.423954 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:43.438494 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:43.440428 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:43.440544 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:43.444712 14015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34433
I20260808 07:10:43.444787 14112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34433 every 8 connection(s)
I20260808 07:10:43.445148 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:43.446142 14113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:43.446714 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14015
I20260808 07:10:43.446813 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:43.447075 18685 external_mini_cluster.cc:1428] Setting key ba60ddc4318405502df8e30480c5e92f
I20260808 07:10:43.448926 14113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf: Bootstrap starting.
I20260808 07:10:43.449651 14113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:43.449939 14113 log.cc:824] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:43.450695 14113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf: No bootstrap required, opened a new log
I20260808 07:10:43.452445 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:43.447606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:58813 (local address 127.0.0.1:34433)
0808 07:10:43.447801 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:43.447810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:43.447844 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:10:43.448253 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:43.448259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:43.448305 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:43.448457 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:43.448668 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:43.449671 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:43.450533 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:43.450985 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:43.451108 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:43.451841 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:43.451844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:43.451845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:43.452004 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:43.452034 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:43.452045 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:43.452139 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:43.452237 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:10:43.452266 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260808 07:10:43.452917 14113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } }
I20260808 07:10:43.453123 14113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:43.453178 14113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f8831d396b4e66b2f3622ced843abf, State: Initialized, Role: FOLLOWER
I20260808 07:10:43.453325 14113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } }
I20260808 07:10:43.453449 14113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:43.453513 14113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:43.453575 14113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:43.453625 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:43.454135 14113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } }
I20260808 07:10:43.454223 14113 leader_election.cc:302] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11f8831d396b4e66b2f3622ced843abf; no voters:
I20260808 07:10:43.454454 14113 leader_election.cc:288] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:43.454488 14118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:43.454615 14118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [term 1 LEADER]: Becoming Leader. State: Replica: 11f8831d396b4e66b2f3622ced843abf, State: Running, Role: LEADER
I20260808 07:10:43.454730 14113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:43.454753 14118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } }
I20260808 07:10:43.455155 14118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f8831d396b4e66b2f3622ced843abf. Latest consensus state: current_term: 1 leader_uuid: "11f8831d396b4e66b2f3622ced843abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } } }
I20260808 07:10:43.455184 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11f8831d396b4e66b2f3622ced843abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f8831d396b4e66b2f3622ced843abf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34433 } } }
I20260808 07:10:43.455329 14118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:43.455365 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:43.455871 14133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:43.456640 14133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:43.459090 14133 catalog_manager.cc:1347] Generated new cluster ID: 88311a580f8345e681e9e8f79e4a7e0f
I20260808 07:10:43.459147 14133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:43.460088 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:43.456191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:53611 (local address 127.0.0.1:34433)
0808 07:10:43.456231 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:43.456233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:43.456433 (+ 200us) server_negotiation.cc:407] Connection header received
0808 07:10:43.456545 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:43.456546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:43.456557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:43.456622 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:43.456984 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:43.457729 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:43.458717 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:43.458926 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:43.459017 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:43.459824 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:43.459825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:43.459826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:43.459830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:43.459846 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:43.459850 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:43.459915 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:43.459989 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 07:10:43.459999 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:43.469192 14133 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:43.469763 14133 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:43.472975 14133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11f8831d396b4e66b2f3622ced843abf: Generated new TSK 0
I20260808 07:10:43.473151 14133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:43.481918 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14015
W20260808 07:10:43.498943 13995 connection.cc:535] client connection to 127.0.0.1:34433 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34433): Connection reset by peer (error 104)
2026-08-08T07:10:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:43 dist-test-slave-bw88 krb5kdc[14148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:43 dist-test-slave-bw88 krb5kdc[14148](info): set up 2 sockets
Aug 08 07:10:43 dist-test-slave-bw88 krb5kdc[14148](info): commencing operation
krb5kdc: starting...
W20260808 07:10:44.677556 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:44 dist-test-slave-bw88 krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:44.808529 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:44.900583 14168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:44.900856 14168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:44.900902 14168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:44.902287 14168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:44.902364 14168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:44.902395 14168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:44.902428 14168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:44.902456 14168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:44.904129 14168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:44.904441 14168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:44.904706 14168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:44.910303 14176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:44.910352 14177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:44.910362 14180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:44.910418 14168 server_base.cc:1034] running on GCE node
I20260808 07:10:44.910786 14168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:44.911118 14168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:44.912331 14168 hybrid_clock.cc:648] HybridClock initialized: now 1786173044912305 us; error 43 us; skew 500 ppm
Aug 08 07:10:44 dist-test-slave-bw88 krb5kdc[14148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:44.920425 14168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:44.920892 14168 webserver.cc:460] Webserver started at http://127.0.0.1:36949/ using document root <none> and password file <none>
I20260808 07:10:44.921134 14168 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:44.921221 14168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:44.921429 14168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:44.922536 14168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "23dc6c66e1e0411d889baa320b49e9ce"
format_stamp: "Formatted at 2026-08-08 07:10:44 on dist-test-slave-bw88"
server_key: "d1875fb3dc3673fb20aa8f577dcddb3f"
server_key_iv: "2ec6bb0e9ec848f1b226db80136bed8c"
server_key_version: "encryptionkey@0"
I20260808 07:10:44.922909 14168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "23dc6c66e1e0411d889baa320b49e9ce"
format_stamp: "Formatted at 2026-08-08 07:10:44 on dist-test-slave-bw88"
server_key: "d1875fb3dc3673fb20aa8f577dcddb3f"
server_key_iv: "2ec6bb0e9ec848f1b226db80136bed8c"
server_key_version: "encryptionkey@0"
I20260808 07:10:44.924279 14168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:44.925045 14191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:44.925241 14168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:44.925372 14168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "23dc6c66e1e0411d889baa320b49e9ce"
format_stamp: "Formatted at 2026-08-08 07:10:44 on dist-test-slave-bw88"
server_key: "d1875fb3dc3673fb20aa8f577dcddb3f"
server_key_iv: "2ec6bb0e9ec848f1b226db80136bed8c"
server_key_version: "encryptionkey@0"
I20260808 07:10:44.925450 14168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:44.928750 14168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:44.931082 14168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:44.931245 14168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:44.935896 14168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38093
I20260808 07:10:44.935984 14263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38093 every 8 connection(s)
I20260808 07:10:44.936267 14168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:44.937173 14265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:44.939497 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce: Bootstrap starting.
I20260808 07:10:44.940033 14265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:44.940310 14265 log.cc:824] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:44.941023 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce: No bootstrap required, opened a new log
I20260808 07:10:44.942685 14265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } }
I20260808 07:10:44.942823 14265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:44.942845 14265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23dc6c66e1e0411d889baa320b49e9ce, State: Initialized, Role: FOLLOWER
I20260808 07:10:44.942945 14265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } }
I20260808 07:10:44.942996 14265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:44.943015 14265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:44.943041 14265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:44.943924 14265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } }
I20260808 07:10:44.944017 14265 leader_election.cc:302] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23dc6c66e1e0411d889baa320b49e9ce; no voters:
I20260808 07:10:44.944142 14265 leader_election.cc:288] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:44.944212 14268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:44.944442 14268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [term 1 LEADER]: Becoming Leader. State: Replica: 23dc6c66e1e0411d889baa320b49e9ce, State: Running, Role: LEADER
I20260808 07:10:44.944448 14265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:44.944526 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14168
I20260808 07:10:44.944659 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:44.944597 14268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } }
I20260808 07:10:44.944861 18685 external_mini_cluster.cc:1428] Setting key fbad7599f61c59d10a80a57d57e7f115
I20260808 07:10:44.944962 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23dc6c66e1e0411d889baa320b49e9ce. Latest consensus state: current_term: 1 leader_uuid: "23dc6c66e1e0411d889baa320b49e9ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } } }
I20260808 07:10:44.945139 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:44.945519 14280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:44.944973 14273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23dc6c66e1e0411d889baa320b49e9ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23dc6c66e1e0411d889baa320b49e9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38093 } } }
I20260808 07:10:44.945700 14273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:44.946189 14280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:44.947687 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:44.945329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:47903 (local address 127.0.0.1:38093)
0808 07:10:44.946065 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:44.946076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:44.946150 (+ 74us) server_negotiation.cc:407] Connection header received
0808 07:10:44.946427 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:44.946435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:44.946469 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:44.946643 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:44.947270 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:44.947274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:44.947274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:44.947278 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:44.947329 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:44.947340 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:44.947415 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:44.947485 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 07:10:44.947531 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":79,"threads_started":1}
I20260808 07:10:44.948827 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:44.951493 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:44.950350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:34983 (local address 127.0.0.1:38093)
0808 07:10:44.950394 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:44.950396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:44.950484 (+ 88us) server_negotiation.cc:407] Connection header received
0808 07:10:44.950510 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:44.950511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:44.950517 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:44.950595 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:44.951239 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:44.951241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:44.951241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:44.951245 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:44.951262 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:44.951266 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:44.951344 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:44.951384 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:10:44.951397 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:10:44.953459 14280 catalog_manager.cc:1347] Generated new cluster ID: 44638d29eb1f4a5c941cdf47008349ae
I20260808 07:10:44.953527 14280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:44.970296 14280 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:44.970496 14280 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:44.979238 14280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23dc6c66e1e0411d889baa320b49e9ce: Generated new TSK 0
I20260808 07:10:44.979458 14280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:44.984879 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14168
2026-08-08T07:10:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:45 dist-test-slave-bw88 krb5kdc[14310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:45 dist-test-slave-bw88 krb5kdc[14310](info): set up 2 sockets
Aug 08 07:10:45 dist-test-slave-bw88 krb5kdc[14310](info): commencing operation
krb5kdc: starting...
W20260808 07:10:46.198053 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:46.323777 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:46.407469 14335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:46.407675 14335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:46.407696 14335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:46.408912 14335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:46.408953 14335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:46.408967 14335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:46.408979 14335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:46.409022 14335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:46.410516 14335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:46.410770 14335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:46.411005 14335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:46.416016 14341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:46.416070 14343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:46.416090 14335 server_base.cc:1034] running on GCE node
W20260808 07:10:46.416067 14345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:46.416445 14335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:46.416783 14335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:46.417948 14335 hybrid_clock.cc:648] HybridClock initialized: now 1786173046417937 us; error 53 us; skew 500 ppm
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173046, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:46.420562 14335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:46.421036 14335 webserver.cc:460] Webserver started at http://127.0.0.1:35061/ using document root <none> and password file <none>
I20260808 07:10:46.421271 14335 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:46.421339 14335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:46.421525 14335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:46.422590 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "0c03af27350644adb9763a857cf319bf"
format_stamp: "Formatted at 2026-08-08 07:10:46 on dist-test-slave-bw88"
server_key: "5a2fbe26d9d5134a978f2b9e9be9f0cc"
server_key_iv: "a47392d4b49c123a25f3b7f60fefae45"
server_key_version: "encryptionkey@0"
I20260808 07:10:46.422958 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "0c03af27350644adb9763a857cf319bf"
format_stamp: "Formatted at 2026-08-08 07:10:46 on dist-test-slave-bw88"
server_key: "5a2fbe26d9d5134a978f2b9e9be9f0cc"
server_key_iv: "a47392d4b49c123a25f3b7f60fefae45"
server_key_version: "encryptionkey@0"
I20260808 07:10:46.424316 14335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:46.425148 14353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:46.425387 14335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:46.425536 14335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "0c03af27350644adb9763a857cf319bf"
format_stamp: "Formatted at 2026-08-08 07:10:46 on dist-test-slave-bw88"
server_key: "5a2fbe26d9d5134a978f2b9e9be9f0cc"
server_key_iv: "a47392d4b49c123a25f3b7f60fefae45"
server_key_version: "encryptionkey@0"
I20260808 07:10:46.425619 14335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:46.449857 14335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:46.451879 14335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:46.452064 14335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:46.456435 14335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34161
I20260808 07:10:46.456507 14423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34161 every 8 connection(s)
I20260808 07:10:46.456887 14335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:46.457944 14425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:46.459501 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14335
I20260808 07:10:46.459657 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:46.459880 18685 external_mini_cluster.cc:1428] Setting key 7005940cf3ff3960bda501b4b1c3dae6
I20260808 07:10:46.461009 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf: Bootstrap starting.
I20260808 07:10:46.461835 14425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:46.462232 14425 log.cc:824] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:46.463065 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf: No bootstrap required, opened a new log
I20260808 07:10:46.465754 14425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } }
I20260808 07:10:46.465962 14425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:46.466008 14425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c03af27350644adb9763a857cf319bf, State: Initialized, Role: FOLLOWER
I20260808 07:10:46.466162 14425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } }
I20260808 07:10:46.466266 14425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:46.466305 14425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:46.466369 14425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:46.467160 14425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } }
I20260808 07:10:46.467257 14425 leader_election.cc:302] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c03af27350644adb9763a857cf319bf; no voters:
I20260808 07:10:46.467485 14425 leader_election.cc:288] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 07:10:46.467537 14433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:46.467748 14433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [term 1 LEADER]: Becoming Leader. State: Replica: 0c03af27350644adb9763a857cf319bf, State: Running, Role: LEADER
I20260808 07:10:46.467783 14425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:46.467931 14433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } }
I20260808 07:10:46.468593 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c03af27350644adb9763a857cf319bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } } }
I20260808 07:10:46.468683 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:46.468930 14435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c03af27350644adb9763a857cf319bf. Latest consensus state: current_term: 1 leader_uuid: "0c03af27350644adb9763a857cf319bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c03af27350644adb9763a857cf319bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34161 } } }
I20260808 07:10:46.469334 14435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:46.469533 14440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:46.470094 14440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:46.472373 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:46.460519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:46211 (local address 127.0.0.1:34161)
0808 07:10:46.460689 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:46.460699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:46.460735 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:10:46.461207 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:46.461218 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:46.461274 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:46.461448 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:46.462376 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:46.463389 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:46.464551 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:46.464946 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:46.465030 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:46.468465 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:46.468469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:46.468470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:46.468517 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:46.470681 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:46.471166 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:46.471169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:46.471169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:46.471295 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:46.471570 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:46.471572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:46.471572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:46.471814 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:46.472008 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:46.472104 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:10:46.472206 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260808 07:10:46.472723 14440 catalog_manager.cc:1347] Generated new cluster ID: 44b5d2a7cc144fa5bc174aac9084bbcd
I20260808 07:10:46.472784 14440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:46.473469 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:46.487695 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:46.474876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:59115 (local address 127.0.0.1:34161)
0808 07:10:46.474912 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:46.474914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:46.481537 (+ 6623us) server_negotiation.cc:407] Connection header received
0808 07:10:46.481650 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:46.481651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:46.481662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:46.481719 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:46.482274 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:46.482807 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:46.483537 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:46.483758 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:46.483836 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:46.485040 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:46.485042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:46.485042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:46.485050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:46.486501 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:46.486876 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:46.486877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:46.486877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:46.486939 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:46.487173 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:46.487174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:46.487175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:46.487288 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:46.487404 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:46.487521 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 07:10:46.487583 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:46.488046 14440 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:46.488775 14440 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:46.496961 14440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c03af27350644adb9763a857cf319bf: Generated new TSK 0
I20260808 07:10:46.497159 14440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:46.499214 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14335
W20260808 07:10:46.519202 14307 connection.cc:535] client connection to 127.0.0.1:34161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34161): Connection reset by peer (error 104)
2026-08-08T07:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14467](info): set up 2 sockets
Aug 08 07:10:46 dist-test-slave-bw88 krb5kdc[14467](info): commencing operation
krb5kdc: starting...
W20260808 07:10:47.720353 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:47 dist-test-slave-bw88 krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:47.852711 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:47.938133 14485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:47.938333 14485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:47.938356 14485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:47.939433 14485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:47.939471 14485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:47.939493 14485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:47.939505 14485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:47.939517 14485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:47.940914 14485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:47.941100 14485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:47.941287 14485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:47.946372 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:47.946409 14485 server_base.cc:1034] running on GCE node
W20260808 07:10:47.946412 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:47.946373 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:47.946777 14485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:47.947062 14485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:47.948204 14485 hybrid_clock.cc:648] HybridClock initialized: now 1786173047948180 us; error 49 us; skew 500 ppm
Aug 08 07:10:47 dist-test-slave-bw88 krb5kdc[14467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173047, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:47.950831 14485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:47.951282 14485 webserver.cc:460] Webserver started at http://127.0.0.1:37715/ using document root <none> and password file <none>
I20260808 07:10:47.951459 14485 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:47.951493 14485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:47.951593 14485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:47.952574 14485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "a4c9a224320c49e882ea9647e23d9991"
format_stamp: "Formatted at 2026-08-08 07:10:47 on dist-test-slave-bw88"
server_key: "0d3a22b21b2f86a3d750aaa8c0a81ad0"
server_key_iv: "d97b0c38e7573732b498319514aa8f1d"
server_key_version: "encryptionkey@0"
I20260808 07:10:47.952880 14485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "a4c9a224320c49e882ea9647e23d9991"
format_stamp: "Formatted at 2026-08-08 07:10:47 on dist-test-slave-bw88"
server_key: "0d3a22b21b2f86a3d750aaa8c0a81ad0"
server_key_iv: "d97b0c38e7573732b498319514aa8f1d"
server_key_version: "encryptionkey@0"
I20260808 07:10:47.954234 14485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:47.954957 14508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:47.955158 14485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:47.955268 14485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "a4c9a224320c49e882ea9647e23d9991"
format_stamp: "Formatted at 2026-08-08 07:10:47 on dist-test-slave-bw88"
server_key: "0d3a22b21b2f86a3d750aaa8c0a81ad0"
server_key_iv: "d97b0c38e7573732b498319514aa8f1d"
server_key_version: "encryptionkey@0"
I20260808 07:10:47.955343 14485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:47.978497 14485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:47.981050 14485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:47.981245 14485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:47.985738 14485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40553
I20260808 07:10:47.985863 14576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40553 every 8 connection(s)
I20260808 07:10:47.986192 14485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:47.987113 14577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:47.988107 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14485
I20260808 07:10:47.988226 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:47.988440 18685 external_mini_cluster.cc:1428] Setting key 271008983105ac89fd7a8082ea8230fa
I20260808 07:10:47.990617 14577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991: Bootstrap starting.
I20260808 07:10:47.991278 14577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:47.991639 14577 log.cc:824] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:47.992413 14577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991: No bootstrap required, opened a new log
I20260808 07:10:47.994266 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:47.988967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:45447 (local address 127.0.0.1:40553)
0808 07:10:47.989166 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:47.989175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:47.989200 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:10:47.989715 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:47.989723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:47.989776 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:47.989890 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:47.990171 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:47.991163 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:47.992243 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:47.992725 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:47.992808 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:47.993596 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:47.993600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:47.993600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:47.993764 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:47.993795 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:47.993806 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:47.993872 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:47.993993 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 07:10:47.994015 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260808 07:10:47.994839 14577 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } }
I20260808 07:10:47.995029 14577 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:47.995105 14577 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c9a224320c49e882ea9647e23d9991, State: Initialized, Role: FOLLOWER
I20260808 07:10:47.995241 14577 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } }
I20260808 07:10:47.995321 14577 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:47.995344 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:47.995363 14577 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:47.995411 14577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:47.996156 14577 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } }
I20260808 07:10:47.996270 14577 leader_election.cc:302] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4c9a224320c49e882ea9647e23d9991; no voters:
I20260808 07:10:47.996447 14577 leader_election.cc:288] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:47.996508 14587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:47.996668 14587 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [term 1 LEADER]: Becoming Leader. State: Replica: a4c9a224320c49e882ea9647e23d9991, State: Running, Role: LEADER
I20260808 07:10:47.996730 14577 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:47.996797 14587 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } }
I20260808 07:10:47.997125 14587 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4c9a224320c49e882ea9647e23d9991. Latest consensus state: current_term: 1 leader_uuid: "a4c9a224320c49e882ea9647e23d9991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } } }
I20260808 07:10:47.997152 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4c9a224320c49e882ea9647e23d9991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c9a224320c49e882ea9647e23d9991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40553 } } }
I20260808 07:10:47.997329 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:47.997589 14587 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:47.997969 14605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:47.998523 14605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:48.000901 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:47.996895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:51839 (local address 127.0.0.1:40553)
0808 07:10:47.996938 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:47.996940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:47.997308 (+ 368us) server_negotiation.cc:407] Connection header received
0808 07:10:47.997471 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:47.997472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:47.997483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:47.997518 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:47.997734 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:47.998513 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:47.999497 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:47.999792 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:47.999867 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:48.000613 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:48.000614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:48.000615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:48.000618 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:48.000632 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:48.000636 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:48.000686 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:48.000782 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:10:48.000809 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 07:10:48.001767 14605 catalog_manager.cc:1347] Generated new cluster ID: 984250a2fef1460fab05b78f381ff385
I20260808 07:10:48.001824 14605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:48.007453 14605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:48.007941 14605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:48.010531 14605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4c9a224320c49e882ea9647e23d9991: Generated new TSK 0
I20260808 07:10:48.010707 14605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:48.015880 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14485
W20260808 07:10:48.026962 14463 connection.cc:535] client connection to 127.0.0.1:40553 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40553): Connection reset by peer (error 104)
2026-08-08T07:10:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:48 dist-test-slave-bw88 krb5kdc[14623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:48 dist-test-slave-bw88 krb5kdc[14623](info): set up 2 sockets
Aug 08 07:10:48 dist-test-slave-bw88 krb5kdc[14623](info): commencing operation
krb5kdc: starting...
W20260808 07:10:49.215205 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:49 dist-test-slave-bw88 krb5kdc[14623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 07:10:49.344198 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:49.427176 14639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:49.427374 14639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:49.427397 14639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:49.428550 14639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:49.428596 14639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:49.428609 14639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:49.428668 14639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:49.428687 14639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:49.430199 14639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:49.430461 14639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:49.430707 14639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:49.435712 14648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:49.435714 14646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:49.435891 14645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:49.436233 14639 server_base.cc:1034] running on GCE node
I20260808 07:10:49.436429 14639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:49.436691 14639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:49.437832 14639 hybrid_clock.cc:648] HybridClock initialized: now 1786173049437817 us; error 31 us; skew 500 ppm
Aug 08 07:10:49 dist-test-slave-bw88 krb5kdc[14623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:49.440514 14639 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 07:10:49.441015 14639 webserver.cc:460] Webserver started at http://127.0.0.1:45307/ using document root <none> and password file <none>
I20260808 07:10:49.441280 14639 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:49.441327 14639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:49.441536 14639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:49.442597 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "5b4b5afaca04426687d0a5794a94a13b"
format_stamp: "Formatted at 2026-08-08 07:10:49 on dist-test-slave-bw88"
server_key: "7f531c719a37767ab91cd46bf8fe321a"
server_key_iv: "b172ae570df074983a1fb89361748d59"
server_key_version: "encryptionkey@0"
I20260808 07:10:49.442958 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "5b4b5afaca04426687d0a5794a94a13b"
format_stamp: "Formatted at 2026-08-08 07:10:49 on dist-test-slave-bw88"
server_key: "7f531c719a37767ab91cd46bf8fe321a"
server_key_iv: "b172ae570df074983a1fb89361748d59"
server_key_version: "encryptionkey@0"
I20260808 07:10:49.444305 14639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:49.445065 14658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:49.445266 14639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:49.445400 14639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "5b4b5afaca04426687d0a5794a94a13b"
format_stamp: "Formatted at 2026-08-08 07:10:49 on dist-test-slave-bw88"
server_key: "7f531c719a37767ab91cd46bf8fe321a"
server_key_iv: "b172ae570df074983a1fb89361748d59"
server_key_version: "encryptionkey@0"
I20260808 07:10:49.445461 14639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:49.448717 14639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:49.450731 14639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:49.450841 14639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:49.455026 14639 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33131
I20260808 07:10:49.455112 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33131 every 8 connection(s)
I20260808 07:10:49.455349 14639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:49.456359 14733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:49.458874 14733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b: Bootstrap starting.
I20260808 07:10:49.459321 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14639
I20260808 07:10:49.459446 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:49.459667 18685 external_mini_cluster.cc:1428] Setting key 5579365bb01d5c509336fe41d2d41830
I20260808 07:10:49.459676 14733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:49.460019 14733 log.cc:824] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:49.460930 14733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b: No bootstrap required, opened a new log
I20260808 07:10:49.462494 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:49.460210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:38171 (local address 127.0.0.1:33131)
0808 07:10:49.460444 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:49.460453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:49.460494 (+ 41us) server_negotiation.cc:407] Connection header received
0808 07:10:49.461121 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:49.461130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:49.461177 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:49.461328 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:49.462036 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:49.462039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:49.462040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:49.462044 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:49.462090 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:49.462104 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:49.462199 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:49.462233 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 07:10:49.462274 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":113,"threads_started":1}
I20260808 07:10:49.463557 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:49.463476 14733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260808 07:10:49.463662 14733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:49.463701 14733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4b5afaca04426687d0a5794a94a13b, State: Initialized, Role: FOLLOWER
I20260808 07:10:49.463812 14733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260808 07:10:49.463892 14733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:49.463920 14733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:49.463963 14733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:49.464725 14733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260808 07:10:49.464856 14733 leader_election.cc:302] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b4b5afaca04426687d0a5794a94a13b; no voters:
I20260808 07:10:49.465040 14733 leader_election.cc:288] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:49.465075 14748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:49.465281 14748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [term 1 LEADER]: Becoming Leader. State: Replica: 5b4b5afaca04426687d0a5794a94a13b, State: Running, Role: LEADER
I20260808 07:10:49.465317 14733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:49.465518 14748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } }
I20260808 07:10:49.465862 14748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b4b5afaca04426687d0a5794a94a13b. Latest consensus state: current_term: 1 leader_uuid: "5b4b5afaca04426687d0a5794a94a13b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } } }
I20260808 07:10:49.465927 14750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b4b5afaca04426687d0a5794a94a13b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b5afaca04426687d0a5794a94a13b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33131 } } }
I20260808 07:10:49.466013 14748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:49.466092 14750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:49.466367 14760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:49.466935 14760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:49.467813 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:49.465218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.189:57185 (local address 127.0.0.1:33131)
0808 07:10:49.465254 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:49.465256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:49.466503 (+ 1247us) server_negotiation.cc:407] Connection header received
0808 07:10:49.466635 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:49.466636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:49.466642 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:49.466708 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:49.467568 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:49.467569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:49.467570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:49.467574 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:49.467588 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:49.467593 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:49.467623 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:49.467723 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 07:10:49.467737 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:10:49.468662 14760 catalog_manager.cc:1347] Generated new cluster ID: 415f4fae5c224bb699b75e0494749b3d
I20260808 07:10:49.468715 14760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:49.482023 14760 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:49.482214 14760 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:49.484923 14760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b4b5afaca04426687d0a5794a94a13b: Generated new TSK 0
I20260808 07:10:49.485131 14760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:49.492671 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14639
2026-08-08T07:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:49 dist-test-slave-bw88 krb5kdc[14776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:49 dist-test-slave-bw88 krb5kdc[14776](info): set up 2 sockets
Aug 08 07:10:49 dist-test-slave-bw88 krb5kdc[14776](info): commencing operation
krb5kdc: starting...
W20260808 07:10:50.684525 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:50 dist-test-slave-bw88 krb5kdc[14776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:50.811678 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38829
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:38829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:50.898993 14796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:50.899188 14796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:50.899211 14796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:50.900346 14796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:50.900388 14796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:50.900403 14796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:50.900416 14796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:50.900468 14796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:50.901952 14796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:38829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:38829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:50.902209 14796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:50.902451 14796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:50.907672 14803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:50.907712 14802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:50.907758 14805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:50.908185 14796 server_base.cc:1034] running on GCE node
I20260808 07:10:50.908408 14796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:50.908676 14796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:50.909828 14796 hybrid_clock.cc:648] HybridClock initialized: now 1786173050909817 us; error 29 us; skew 500 ppm
Aug 08 07:10:50 dist-test-slave-bw88 krb5kdc[14776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173050, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:50.918321 14796 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:50.918830 14796 webserver.cc:460] Webserver started at http://127.18.63.126:37241/ using document root <none> and password file <none>
I20260808 07:10:50.919055 14796 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:50.919122 14796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:50.919273 14796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:50.920317 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "21386d65633f450792016d74fe8b4b87"
format_stamp: "Formatted at 2026-08-08 07:10:50 on dist-test-slave-bw88"
server_key: "8dab612783741c6739025b45bdb1f5cf"
server_key_iv: "fd8c3fabf34a8c638fcae79003b935da"
server_key_version: "encryptionkey@0"
I20260808 07:10:50.920689 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "21386d65633f450792016d74fe8b4b87"
format_stamp: "Formatted at 2026-08-08 07:10:50 on dist-test-slave-bw88"
server_key: "8dab612783741c6739025b45bdb1f5cf"
server_key_iv: "fd8c3fabf34a8c638fcae79003b935da"
server_key_version: "encryptionkey@0"
I20260808 07:10:50.922224 14796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:50.923032 14813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:50.923236 14796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:50.923353 14796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "21386d65633f450792016d74fe8b4b87"
format_stamp: "Formatted at 2026-08-08 07:10:50 on dist-test-slave-bw88"
server_key: "8dab612783741c6739025b45bdb1f5cf"
server_key_iv: "fd8c3fabf34a8c638fcae79003b935da"
server_key_version: "encryptionkey@0"
I20260808 07:10:50.923444 14796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:50.948959 14796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:50.951308 14796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:50.951474 14796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:50.955642 14796 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:38829
I20260808 07:10:50.955679 14884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:38829 every 8 connection(s)
I20260808 07:10:50.956046 14796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:50.956952 14885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:50.957702 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14796
I20260808 07:10:50.957834 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:50.958068 18685 external_mini_cluster.cc:1428] Setting key a7814b0da95e364d1328716f979bdfe5
I20260808 07:10:50.960079 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87: Bootstrap starting.
I20260808 07:10:50.960664 14885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:50.960968 14885 log.cc:824] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:50.961783 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87: No bootstrap required, opened a new log
I20260808 07:10:50.963714 14885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } }
I20260808 07:10:50.963914 14885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:50.963958 14885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21386d65633f450792016d74fe8b4b87, State: Initialized, Role: FOLLOWER
I20260808 07:10:50.964092 14885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } }
I20260808 07:10:50.964183 14885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:50.964221 14885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:50.964273 14885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 07:10:50 dist-test-slave-bw88 krb5kdc[14776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:50.964975 14885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } }
I20260808 07:10:50.965131 14885 leader_election.cc:302] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21386d65633f450792016d74fe8b4b87; no voters:
I20260808 07:10:50.965308 14885 leader_election.cc:288] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:50.965396 14892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:50.965551 14892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [term 1 LEADER]: Becoming Leader. State: Replica: 21386d65633f450792016d74fe8b4b87, State: Running, Role: LEADER
I20260808 07:10:50.965607 14885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:50.965692 14892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } }
I20260808 07:10:50.966110 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21386d65633f450792016d74fe8b4b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } } }
I20260808 07:10:50.966244 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:50.966347 14896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21386d65633f450792016d74fe8b4b87. Latest consensus state: current_term: 1 leader_uuid: "21386d65633f450792016d74fe8b4b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21386d65633f450792016d74fe8b4b87" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 38829 } } }
I20260808 07:10:50.966478 14896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:50.966696 14905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:50.967222 14905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:50.969004 14905 catalog_manager.cc:1347] Generated new cluster ID: 2502eaf96c1f4f3485d27099bc9f2409
I20260808 07:10:50.969063 14905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:50.969982 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.958553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35745 (local address 127.18.63.126:38829)
0808 07:10:50.958711 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:50.958720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:50.958759 (+ 39us) server_negotiation.cc:407] Connection header received
0808 07:10:50.959210 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:50.959216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:50.959259 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:50.959410 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:50.960050 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:50.961021 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:50.961873 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:50.962280 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:50.962397 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:50.965066 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:50.965070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:50.965070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:50.965080 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:50.968164 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:50.968661 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:50.968664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:50.968664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:50.968768 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:50.969045 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:50.969046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:50.969047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:50.969376 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:50.969535 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:50.969676 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 07:10:50.969772 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260808 07:10:50.970945 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:50.976527 14905 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:50.977228 14905 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:50.980824 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:50.972384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36177 (local address 127.18.63.126:38829)
0808 07:10:50.972415 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:50.972417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:50.973533 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 07:10:50.973644 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:50.973645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:50.973657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:50.973723 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:50.974315 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:50.975098 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:50.976285 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:50.976606 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:50.976682 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:50.978048 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:50.978049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:50.978049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:50.978057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:50.979624 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:50.980035 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:50.980036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:50.980036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:50.980097 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:50.980323 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:50.980324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:50.980324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:50.980450 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:50.980560 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:50.980674 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 07:10:50.980737 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 07:10:50.984876 14905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21386d65633f450792016d74fe8b4b87: Generated new TSK 0
I20260808 07:10:50.985064 14905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:50.993012 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14796
W20260808 07:10:51.004658 14772 connection.cc:535] client connection to 127.18.63.126:38829 recv error: Network error: failed to read from TLS socket (remote: 127.18.63.126:38829): Connection reset by peer (error 104)
2026-08-08T07:10:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:51 dist-test-slave-bw88 krb5kdc[14926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:51 dist-test-slave-bw88 krb5kdc[14926](info): set up 2 sockets
Aug 08 07:10:51 dist-test-slave-bw88 krb5kdc[14926](info): commencing operation
krb5kdc: starting...
W20260808 07:10:52.207922 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:52 dist-test-slave-bw88 krb5kdc[14926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:52.334512 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33199
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:33199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:52.418258 14944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:52.418511 14944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:52.418553 14944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:52.419695 14944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:52.419763 14944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:52.419788 14944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:52.419818 14944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:52.419845 14944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:52.421357 14944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:33199
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:33199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:52.421643 14944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:52.421869 14944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:52.426822 14953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:52.426831 14955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:52.426899 14952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:52.427063 14944 server_base.cc:1034] running on GCE node
I20260808 07:10:52.427289 14944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:52.427608 14944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:52.428785 14944 hybrid_clock.cc:648] HybridClock initialized: now 1786173052428780 us; error 38 us; skew 500 ppm
Aug 08 07:10:52 dist-test-slave-bw88 krb5kdc[14926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173052, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:52.431299 14944 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:52.431799 14944 webserver.cc:460] Webserver started at http://127.18.63.126:44809/ using document root <none> and password file <none>
I20260808 07:10:52.432044 14944 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:52.432113 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:52.432265 14944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:52.433272 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "55f97d00ca304d1d8c2b16bcce04aa56"
format_stamp: "Formatted at 2026-08-08 07:10:52 on dist-test-slave-bw88"
server_key: "633f571037737c4b3cf12c2b61870b43"
server_key_iv: "73dc18eca3264220991c10184abe0869"
server_key_version: "encryptionkey@0"
I20260808 07:10:52.433720 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "55f97d00ca304d1d8c2b16bcce04aa56"
format_stamp: "Formatted at 2026-08-08 07:10:52 on dist-test-slave-bw88"
server_key: "633f571037737c4b3cf12c2b61870b43"
server_key_iv: "73dc18eca3264220991c10184abe0869"
server_key_version: "encryptionkey@0"
I20260808 07:10:52.435093 14944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:10:52.435827 14966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:52.436014 14944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:52.436126 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "55f97d00ca304d1d8c2b16bcce04aa56"
format_stamp: "Formatted at 2026-08-08 07:10:52 on dist-test-slave-bw88"
server_key: "633f571037737c4b3cf12c2b61870b43"
server_key_iv: "73dc18eca3264220991c10184abe0869"
server_key_version: "encryptionkey@0"
I20260808 07:10:52.436203 14944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:52.456408 14944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:52.458698 14944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:52.458884 14944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:52.463882 14944 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:33199
I20260808 07:10:52.463917 15034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:33199 every 8 connection(s)
I20260808 07:10:52.464299 14944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:52.465245 15035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:52.467700 15035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56: Bootstrap starting.
I20260808 07:10:52.468304 15035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:52.468600 15035 log.cc:824] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:52.469408 15035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56: No bootstrap required, opened a new log
I20260808 07:10:52.470175 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 14944
I20260808 07:10:52.470314 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:52.470503 18685 external_mini_cluster.cc:1428] Setting key 49157d3a1d59566116db06014bad2169
I20260808 07:10:52.471653 15035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } }
I20260808 07:10:52.471827 15035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:52.471866 15035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f97d00ca304d1d8c2b16bcce04aa56, State: Initialized, Role: FOLLOWER
I20260808 07:10:52.472060 15035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } }
I20260808 07:10:52.472180 15035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:52.472225 15035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:52.472278 15035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:52.472817 15035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } }
I20260808 07:10:52.472939 15035 leader_election.cc:302] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f97d00ca304d1d8c2b16bcce04aa56; no voters:
I20260808 07:10:52.473122 15035 leader_election.cc:288] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:52.473169 15043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:52.473359 15043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [term 1 LEADER]: Becoming Leader. State: Replica: 55f97d00ca304d1d8c2b16bcce04aa56, State: Running, Role: LEADER
I20260808 07:10:52.473418 15035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:52.473498 15043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } }
I20260808 07:10:52.473907 15043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55f97d00ca304d1d8c2b16bcce04aa56. Latest consensus state: current_term: 1 leader_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } } }
I20260808 07:10:52.474058 15043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:52.474525 15057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:52.475086 15057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:52.475867 15047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f97d00ca304d1d8c2b16bcce04aa56" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 33199 } } }
I20260808 07:10:52.476025 15047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:52.476603 15057 catalog_manager.cc:1347] Generated new cluster ID: a5d0b8e733fe4b0985e20bd2bf489137
I20260808 07:10:52.476665 15057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:52.477136 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:52.471069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60237 (local address 127.18.63.126:33199)
0808 07:10:52.471249 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:52.471256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:52.471285 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:10:52.471632 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:52.471639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:52.471685 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:52.471794 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:52.472092 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:52.473672 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:52.474884 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:52.475395 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:52.475513 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:52.476337 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:52.476342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:52.476343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:52.476578 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:52.476617 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:52.476632 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:52.476728 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:52.476877 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 07:10:52.476909 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 07:10:52.478233 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:52.482437 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:52.479641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53831 (local address 127.18.63.126:33199)
0808 07:10:52.479680 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:52.479682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:52.479707 (+ 25us) server_negotiation.cc:407] Connection header received
0808 07:10:52.479785 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:52.479785 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:52.479805 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:52.479849 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:52.480010 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:52.480454 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:52.481177 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:52.481472 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:52.481567 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:52.482176 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:52.482177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:52.482178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:52.482182 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:52.482194 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:52.482198 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:52.482252 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:52.482333 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 07:10:52.482346 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:10:52.492683 15057 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:52.493387 15057 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:52.500347 15057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55f97d00ca304d1d8c2b16bcce04aa56: Generated new TSK 0
I20260808 07:10:52.500509 15057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:52.502542 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 14944
W20260808 07:10:52.512796 14924 connection.cc:535] client connection to 127.18.63.126:33199 recv error: Network error: failed to read from TLS socket (remote: 127.18.63.126:33199): Connection reset by peer (error 104)
2026-08-08T07:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:52 dist-test-slave-bw88 krb5kdc[15078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:52 dist-test-slave-bw88 krb5kdc[15078](info): set up 2 sockets
Aug 08 07:10:52 dist-test-slave-bw88 krb5kdc[15078](info): commencing operation
krb5kdc: starting...
W20260808 07:10:53.688868 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:53 dist-test-slave-bw88 krb5kdc[15078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:53.817057 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46643
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:46643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:53.902030 15096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:53.902230 15096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:53.902253 15096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:53.903318 15096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:53.903352 15096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:53.903367 15096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:53.903379 15096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:53.903391 15096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:53.904785 15096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:46643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:46643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:53.904982 15096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:53.905169 15096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:53.910269 15101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:53.910269 15103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:53.910246 15106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:53.910516 15096 server_base.cc:1034] running on GCE node
I20260808 07:10:53.910739 15096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:53.910976 15096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:53.912146 15096 hybrid_clock.cc:648] HybridClock initialized: now 1786173053912128 us; error 29 us; skew 500 ppm
Aug 08 07:10:53 dist-test-slave-bw88 krb5kdc[15078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173053, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:53.914685 15096 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:53.915148 15096 webserver.cc:460] Webserver started at http://127.18.63.126:44483/ using document root <none> and password file <none>
I20260808 07:10:53.915380 15096 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:53.915448 15096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:53.915606 15096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:53.916639 15096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "3eef23d06f0f4e119ab46c279e3e9ca1"
format_stamp: "Formatted at 2026-08-08 07:10:53 on dist-test-slave-bw88"
server_key: "7c529f39c02bf7e6adf17d37ba4b8b2d"
server_key_iv: "6cf6932c5f262d0279725756d1179735"
server_key_version: "encryptionkey@0"
I20260808 07:10:53.917004 15096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "3eef23d06f0f4e119ab46c279e3e9ca1"
format_stamp: "Formatted at 2026-08-08 07:10:53 on dist-test-slave-bw88"
server_key: "7c529f39c02bf7e6adf17d37ba4b8b2d"
server_key_iv: "6cf6932c5f262d0279725756d1179735"
server_key_version: "encryptionkey@0"
I20260808 07:10:53.918432 15096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 07:10:53.919250 15116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:53.919440 15096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:53.919586 15096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "3eef23d06f0f4e119ab46c279e3e9ca1"
format_stamp: "Formatted at 2026-08-08 07:10:53 on dist-test-slave-bw88"
server_key: "7c529f39c02bf7e6adf17d37ba4b8b2d"
server_key_iv: "6cf6932c5f262d0279725756d1179735"
server_key_version: "encryptionkey@0"
I20260808 07:10:53.919668 15096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:53.923036 15096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:53.924968 15096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:53.925129 15096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:53.929311 15096 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:46643
I20260808 07:10:53.929376 15190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:46643 every 8 connection(s)
I20260808 07:10:53.929709 15096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:53.930716 15191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:53.931952 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15096
I20260808 07:10:53.932093 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:53.932282 18685 external_mini_cluster.cc:1428] Setting key 5678b513ea01ddcc87db571d9061a107
I20260808 07:10:53.933475 15191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1: Bootstrap starting.
I20260808 07:10:53.934235 15191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:53.934593 15191 log.cc:824] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:53.935214 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:53.932856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49439 (local address 127.18.63.126:46643)
0808 07:10:53.933052 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:53.933061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:53.933095 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:10:53.933655 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:53.933663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:53.933691 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:53.933808 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:53.934556 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:53.934559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:53.934559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:53.934563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:53.934599 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:53.934610 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:53.934663 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:53.934803 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:10:53.934998 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260808 07:10:53.935489 15191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1: No bootstrap required, opened a new log
I20260808 07:10:53.936264 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:53.938220 15191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } }
I20260808 07:10:53.938392 15191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:53.938441 15191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eef23d06f0f4e119ab46c279e3e9ca1, State: Initialized, Role: FOLLOWER
I20260808 07:10:53.938663 15191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } }
I20260808 07:10:53.938781 15191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:53.938820 15191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:53.938875 15191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:53.938997 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:53.937896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54393 (local address 127.18.63.126:46643)
0808 07:10:53.937938 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:53.937940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:53.938034 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:10:53.938102 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:53.938103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:53.938106 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:53.938152 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:53.938725 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:53.938726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:53.938726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:53.938730 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:53.938741 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:53.938745 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:53.938827 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:53.938919 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 07:10:53.938932 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:10:53.939524 15191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } }
I20260808 07:10:53.939620 15191 leader_election.cc:302] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eef23d06f0f4e119ab46c279e3e9ca1; no voters:
I20260808 07:10:53.939736 15191 leader_election.cc:288] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:53.939805 15201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:53.939957 15201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 3eef23d06f0f4e119ab46c279e3e9ca1, State: Running, Role: LEADER
I20260808 07:10:53.939970 15191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:53.940076 15201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } }
I20260808 07:10:53.940428 15202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } } }
I20260808 07:10:53.940474 15203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eef23d06f0f4e119ab46c279e3e9ca1. Latest consensus state: current_term: 1 leader_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eef23d06f0f4e119ab46c279e3e9ca1" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 46643 } } }
I20260808 07:10:53.940532 15202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:53.940562 15203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:53.940744 15210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:53.941391 15210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:53.943035 15210 catalog_manager.cc:1347] Generated new cluster ID: f45bfeea9f874e069263aeb27245b990
I20260808 07:10:53.943092 15210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:53.963320 15210 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:53.963515 15210 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:53.970321 15210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eef23d06f0f4e119ab46c279e3e9ca1: Generated new TSK 0
I20260808 07:10:53.970515 15210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:53.980643 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15096
2026-08-08T07:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:54 dist-test-slave-bw88 krb5kdc[15227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:54 dist-test-slave-bw88 krb5kdc[15227](info): set up 2 sockets
Aug 08 07:10:54 dist-test-slave-bw88 krb5kdc[15227](info): commencing operation
krb5kdc: starting...
W20260808 07:10:55.191540 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:55.324427 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37777
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:55.408310 15245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:55.408551 15245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:55.408588 15245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:55.409807 15245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:55.409869 15245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:55.409895 15245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:55.409955 15245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:55.409987 15245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:55.411440 15245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:55.411722 15245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:55.411945 15245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:55.417109 15263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:55.417146 15245 server_base.cc:1034] running on GCE node
W20260808 07:10:55.417127 15259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:55.417115 15258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:55.417627 15245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:55.417934 15245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:55.419077 15245 hybrid_clock.cc:648] HybridClock initialized: now 1786173055419053 us; error 48 us; skew 500 ppm
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173055, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:55.421710 15245 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:55.422163 15245 webserver.cc:460] Webserver started at http://127.18.63.126:39443/ using document root <none> and password file <none>
I20260808 07:10:55.422394 15245 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:55.422441 15245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:55.422616 15245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:55.423673 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "04085023ad064f57a74e3d3b9a6223a3"
format_stamp: "Formatted at 2026-08-08 07:10:55 on dist-test-slave-bw88"
server_key: "187fa4e38419b43df42e709387eb5fe5"
server_key_iv: "dc33110845253e49514305c779b2512f"
server_key_version: "encryptionkey@0"
I20260808 07:10:55.424068 15245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "04085023ad064f57a74e3d3b9a6223a3"
format_stamp: "Formatted at 2026-08-08 07:10:55 on dist-test-slave-bw88"
server_key: "187fa4e38419b43df42e709387eb5fe5"
server_key_iv: "dc33110845253e49514305c779b2512f"
server_key_version: "encryptionkey@0"
I20260808 07:10:55.425487 15245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:55.426266 15271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:55.426451 15245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 07:10:55.426554 15245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "04085023ad064f57a74e3d3b9a6223a3"
format_stamp: "Formatted at 2026-08-08 07:10:55 on dist-test-slave-bw88"
server_key: "187fa4e38419b43df42e709387eb5fe5"
server_key_iv: "dc33110845253e49514305c779b2512f"
server_key_version: "encryptionkey@0"
I20260808 07:10:55.426625 15245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:55.443660 15245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:55.445760 15245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:55.445931 15245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:55.450119 15245 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:37777
I20260808 07:10:55.450194 15344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:37777 every 8 connection(s)
I20260808 07:10:55.450554 15245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:55.451467 15345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:55.454025 15345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: Bootstrap starting.
I20260808 07:10:55.454663 15345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:55.454969 15345 log.cc:824] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:55.455794 15345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: No bootstrap required, opened a new log
I20260808 07:10:55.457465 15345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } }
I20260808 07:10:55.457608 15345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:55.457631 15345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04085023ad064f57a74e3d3b9a6223a3, State: Initialized, Role: FOLLOWER
I20260808 07:10:55.457818 15345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } }
I20260808 07:10:55.457885 15345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:55.457926 15345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:55.457978 15345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:55.458844 15345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } }
I20260808 07:10:55.458959 15345 leader_election.cc:302] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04085023ad064f57a74e3d3b9a6223a3; no voters:
I20260808 07:10:55.459151 15345 leader_election.cc:288] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:55.459190 15349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:55.459367 15349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [term 1 LEADER]: Becoming Leader. State: Replica: 04085023ad064f57a74e3d3b9a6223a3, State: Running, Role: LEADER
I20260808 07:10:55.459451 15345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:55.459533 15349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } }
I20260808 07:10:55.460002 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15245
I20260808 07:10:55.460143 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:55.460376 18685 external_mini_cluster.cc:1428] Setting key 32558ec9ae339e17de045ab9adc175cf
I20260808 07:10:55.460831 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04085023ad064f57a74e3d3b9a6223a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } } }
I20260808 07:10:55.461040 15353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04085023ad064f57a74e3d3b9a6223a3. Latest consensus state: current_term: 1 leader_uuid: "04085023ad064f57a74e3d3b9a6223a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04085023ad064f57a74e3d3b9a6223a3" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37777 } } }
I20260808 07:10:55.461145 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:55.461463 15353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3 [sys.catalog]: This master's current role is: LEADER
W20260808 07:10:55.461814 15370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:10:55.461875 15370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:10:55.461938 15373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:55.462139 15373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:55.463650 15373 catalog_manager.cc:1347] Generated new cluster ID: c6c63eb2618d47de977f581526d9309a
I20260808 07:10:55.463707 15373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:10:55.472570 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:55.461084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33885 (local address 127.18.63.126:37777)
0808 07:10:55.461271 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:55.461283 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:55.461317 (+ 34us) server_negotiation.cc:407] Connection header received
0808 07:10:55.461464 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:55.461472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:55.461557 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:55.461714 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:55.462579 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:55.463988 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:55.465020 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:55.465693 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:55.465814 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:55.468819 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:55.468823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:55.468823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:55.468836 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:55.470854 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:55.471279 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:55.471282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:55.471282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:55.471397 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:55.471627 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:55.471628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:55.471628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:55.471930 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:55.472079 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:55.472204 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 07:10:55.472337 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260808 07:10:55.473506 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:55.482160 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:55.474994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48789 (local address 127.18.63.126:37777)
0808 07:10:55.475030 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:55.475031 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:55.475164 (+ 133us) server_negotiation.cc:407] Connection header received
0808 07:10:55.475212 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:55.475213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:55.475239 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:55.475290 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:55.475900 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:55.476634 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:55.477669 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:55.477972 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:55.478038 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:55.479387 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:55.479388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:55.479388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:10:55.479394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:55.480899 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:55.481336 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:55.481338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:55.481338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:55.481418 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:10:55.481633 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:10:55.481634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:10:55.481634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:10:55.481760 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:55.481856 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:55.481996 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 07:10:55.482073 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 07:10:55.495270 15373 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:55.495843 15373 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:55.500674 15373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04085023ad064f57a74e3d3b9a6223a3: Generated new TSK 0
I20260808 07:10:55.500869 15373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:55.505787 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15245
2026-08-08T07:10:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15384](info): set up 2 sockets
Aug 08 07:10:55 dist-test-slave-bw88 krb5kdc[15384](info): commencing operation
krb5kdc: starting...
W20260808 07:10:56.698052 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:56 dist-test-slave-bw88 krb5kdc[15384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:56.827894 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37635
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:37635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:56.914913 15402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:56.915158 15402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:56.915198 15402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:56.916649 15402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:56.916743 15402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:56.916772 15402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:56.916802 15402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:56.916834 15402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:56.918399 15402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:37635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:56.918695 15402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:56.918931 15402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:56.924204 15413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:56.924194 15410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:56.924244 15402 server_base.cc:1034] running on GCE node
W20260808 07:10:56.924196 15409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:56.924700 15402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:56.924952 15402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:56.926101 15402 hybrid_clock.cc:648] HybridClock initialized: now 1786173056926077 us; error 44 us; skew 500 ppm
Aug 08 07:10:56 dist-test-slave-bw88 krb5kdc[15384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173056, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:56.932206 15402 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:56.932710 15402 webserver.cc:460] Webserver started at http://127.18.63.126:39569/ using document root <none> and password file <none>
I20260808 07:10:56.932928 15402 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:56.932972 15402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:56.933074 15402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:56.934180 15402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "2b30e9835cf44e849ad932d1e6c33f10"
format_stamp: "Formatted at 2026-08-08 07:10:56 on dist-test-slave-bw88"
server_key: "68e29b28af207d619d372577765b9e71"
server_key_iv: "9672f51d46456e69acd4809c3940b99d"
server_key_version: "encryptionkey@0"
I20260808 07:10:56.934520 15402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "2b30e9835cf44e849ad932d1e6c33f10"
format_stamp: "Formatted at 2026-08-08 07:10:56 on dist-test-slave-bw88"
server_key: "68e29b28af207d619d372577765b9e71"
server_key_iv: "9672f51d46456e69acd4809c3940b99d"
server_key_version: "encryptionkey@0"
I20260808 07:10:56.935815 15402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:56.936532 15421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:56.936739 15402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:56.936863 15402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "2b30e9835cf44e849ad932d1e6c33f10"
format_stamp: "Formatted at 2026-08-08 07:10:56 on dist-test-slave-bw88"
server_key: "68e29b28af207d619d372577765b9e71"
server_key_iv: "9672f51d46456e69acd4809c3940b99d"
server_key_version: "encryptionkey@0"
I20260808 07:10:56.936944 15402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:56.953713 15402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:56.955668 15402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:56.955837 15402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:56.960060 15402 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:37635
I20260808 07:10:56.960103 15496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:37635 every 8 connection(s)
I20260808 07:10:56.960471 15402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:56.961439 15497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:56.963258 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15402
I20260808 07:10:56.963392 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:56.963591 18685 external_mini_cluster.cc:1428] Setting key 42c8b102850a574bb71d0f5d5c71b45b
I20260808 07:10:56.963877 15497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10: Bootstrap starting.
I20260808 07:10:56.964676 15497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:56.965036 15497 log.cc:824] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:56.965871 15497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10: No bootstrap required, opened a new log
I20260808 07:10:56.967672 15497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } }
I20260808 07:10:56.967840 15497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:56.967933 15497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b30e9835cf44e849ad932d1e6c33f10, State: Initialized, Role: FOLLOWER
I20260808 07:10:56.968124 15497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } }
I20260808 07:10:56.968226 15497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:56.968272 15497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:56.968333 15497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:56.968930 15497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } }
I20260808 07:10:56.969053 15497 leader_election.cc:302] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b30e9835cf44e849ad932d1e6c33f10; no voters:
I20260808 07:10:56.969250 15497 leader_election.cc:288] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:56.969273 15502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:56.969468 15502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [term 1 LEADER]: Becoming Leader. State: Replica: 2b30e9835cf44e849ad932d1e6c33f10, State: Running, Role: LEADER
I20260808 07:10:56.969522 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.964159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59971 (local address 127.18.63.126:37635)
0808 07:10:56.964391 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:56.964401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:56.964436 (+ 35us) server_negotiation.cc:407] Connection header received
0808 07:10:56.965105 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:56.965114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:56.965177 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:56.965328 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:56.965596 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:56.966568 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:56.967397 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:56.967817 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:56.967941 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:56.968705 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:56.968709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:56.968709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:56.968884 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:56.968914 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:56.968925 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:56.969005 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:56.969195 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:10:56.969268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260808 07:10:56.969552 15497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:56.969621 15502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } }
I20260808 07:10:56.970551 15504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b30e9835cf44e849ad932d1e6c33f10. Latest consensus state: current_term: 1 leader_uuid: "2b30e9835cf44e849ad932d1e6c33f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } }
I20260808 07:10:56.970677 15504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:56.970723 15503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b30e9835cf44e849ad932d1e6c33f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b30e9835cf44e849ad932d1e6c33f10" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 37635 } } }
I20260808 07:10:56.970791 15503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:56.970929 15513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:56.971084 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:56.971439 15513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:56.974452 15513 catalog_manager.cc:1347] Generated new cluster ID: ca6ef7916dab480f963da14cedb2bce4
I20260808 07:10:56.974514 15513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:56.977494 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:56.972638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46261 (local address 127.18.63.126:37635)
0808 07:10:56.972675 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:56.972677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:56.974138 (+ 1461us) server_negotiation.cc:407] Connection header received
0808 07:10:56.974160 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:56.974161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:56.974170 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:56.974218 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:56.974430 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:56.975150 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:56.976194 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:10:56.976493 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:10:56.976551 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:10:56.977221 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:56.977222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:56.977223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:56.977227 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:56.977241 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:56.977245 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:56.977298 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:56.977399 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 07:10:56.977418 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:10:56.980680 15513 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:56.981238 15513 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:56.992970 15513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b30e9835cf44e849ad932d1e6c33f10: Generated new TSK 0
I20260808 07:10:56.993170 15513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:57.001919 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15402
2026-08-08T07:10:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:57 dist-test-slave-bw88 krb5kdc[15534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:57 dist-test-slave-bw88 krb5kdc[15534](info): set up 2 sockets
Aug 08 07:10:57 dist-test-slave-bw88 krb5kdc[15534](info): commencing operation
krb5kdc: starting...
W20260808 07:10:58.194468 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:58 dist-test-slave-bw88 krb5kdc[15534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:58.320156 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:39421
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:33329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:58.403754 15555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:58.404024 15555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:58.404063 15555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:58.405192 15555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:58.405270 15555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:58.405295 15555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:58.405323 15555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:58.405342 15555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:58.406900 15555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:33329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:39421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:58.407176 15555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:58.407418 15555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:58.412488 15561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:58.412573 15555 server_base.cc:1034] running on GCE node
W20260808 07:10:58.412501 15560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:58.412521 15563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:58.412989 15555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:58.413293 15555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:58.414475 15555 hybrid_clock.cc:648] HybridClock initialized: now 1786173058414452 us; error 41 us; skew 500 ppm
Aug 08 07:10:58 dist-test-slave-bw88 krb5kdc[15534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173058, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:58.417105 15555 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:58.417578 15555 webserver.cc:460] Webserver started at http://127.18.63.126:33749/ using document root <none> and password file <none>
I20260808 07:10:58.417838 15555 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:58.417906 15555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:58.418054 15555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:58.419160 15555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "8add510d3d0840f09f91087c0f42add0"
format_stamp: "Formatted at 2026-08-08 07:10:58 on dist-test-slave-bw88"
server_key: "e3c5aaff2ce877d3f49cfeec63b9a04f"
server_key_iv: "e809b5e6416be1ae3e7d0953d0d9c458"
server_key_version: "encryptionkey@0"
I20260808 07:10:58.419512 15555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "8add510d3d0840f09f91087c0f42add0"
format_stamp: "Formatted at 2026-08-08 07:10:58 on dist-test-slave-bw88"
server_key: "e3c5aaff2ce877d3f49cfeec63b9a04f"
server_key_iv: "e809b5e6416be1ae3e7d0953d0d9c458"
server_key_version: "encryptionkey@0"
I20260808 07:10:58.420894 15555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:10:58.421751 15574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:58.422012 15555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:10:58.422145 15555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "8add510d3d0840f09f91087c0f42add0"
format_stamp: "Formatted at 2026-08-08 07:10:58 on dist-test-slave-bw88"
server_key: "e3c5aaff2ce877d3f49cfeec63b9a04f"
server_key_iv: "e809b5e6416be1ae3e7d0953d0d9c458"
server_key_version: "encryptionkey@0"
I20260808 07:10:58.422224 15555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:10:58.425601 15555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:10:58.427841 15555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:10:58.428028 15555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:10:58.432545 15555 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:39421
I20260808 07:10:58.432591 15646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:39421 every 8 connection(s)
I20260808 07:10:58.432945 15555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:10:58.434132 15647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:10:58.435791 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15555
I20260808 07:10:58.435932 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:10:58.436136 18685 external_mini_cluster.cc:1428] Setting key c9ef80d506c25df9deb6d4c649938a65
I20260808 07:10:58.436946 15647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0: Bootstrap starting.
I20260808 07:10:58.437748 15647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0: Neither blocks nor log segments found. Creating new log.
I20260808 07:10:58.438532 15647 log.cc:824] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0: Log is configured to *not* fsync() on all Append() calls
I20260808 07:10:58.438795 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:58.436680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47781 (local address 127.18.63.126:39421)
0808 07:10:58.436866 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:58.436874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:58.436898 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:10:58.437444 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:58.437451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:58.437478 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:58.437608 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:58.438373 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:58.438376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:58.438376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:58.438379 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:58.438411 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:58.438421 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:58.438489 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:58.438578 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 07:10:58.438631 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260808 07:10:58.439373 15647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0: No bootstrap required, opened a new log
I20260808 07:10:58.440021 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:10:58.441557 15647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } }
I20260808 07:10:58.441756 15647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 07:10:58.441799 15647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8add510d3d0840f09f91087c0f42add0, State: Initialized, Role: FOLLOWER
I20260808 07:10:58.441934 15647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } }
I20260808 07:10:58.442018 15647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 07:10:58.442044 15647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 07:10:58.442109 15647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:10:58.442763 15652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:10:58.441554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37193 (local address 127.18.63.126:39421)
0808 07:10:58.441617 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:10:58.441619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:10:58.441684 (+ 65us) server_negotiation.cc:407] Connection header received
0808 07:10:58.441763 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:10:58.441764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:10:58.441769 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:10:58.441828 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:10:58.442533 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:10:58.442535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:10:58.442535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 07:10:58.442540 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:10:58.442554 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 07:10:58.442558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:10:58.442628 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 07:10:58.442672 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 07:10:58.442686 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 07:10:58.442755 15647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } }
I20260808 07:10:58.442885 15647 leader_election.cc:302] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8add510d3d0840f09f91087c0f42add0; no voters:
I20260808 07:10:58.443060 15647 leader_election.cc:288] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 07:10:58.443081 15664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:10:58.443243 15664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [term 1 LEADER]: Becoming Leader. State: Replica: 8add510d3d0840f09f91087c0f42add0, State: Running, Role: LEADER
I20260808 07:10:58.443357 15647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:10:58.443354 15664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } }
I20260808 07:10:58.443691 15664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8add510d3d0840f09f91087c0f42add0. Latest consensus state: current_term: 1 leader_uuid: "8add510d3d0840f09f91087c0f42add0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } } }
I20260808 07:10:58.443776 15664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:58.443702 15665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8add510d3d0840f09f91087c0f42add0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add510d3d0840f09f91087c0f42add0" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 39421 } } }
I20260808 07:10:58.443883 15665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0 [sys.catalog]: This master's current role is: LEADER
I20260808 07:10:58.444057 15673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:10:58.444792 15673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:10:58.446445 15673 catalog_manager.cc:1347] Generated new cluster ID: 8ed01595e3d4420d8c34bffb7d216c5b
I20260808 07:10:58.446489 15673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:10:58.454334 15673 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:10:58.454504 15673 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:10:58.462601 15673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8add510d3d0840f09f91087c0f42add0: Generated new TSK 0
I20260808 07:10:58.462787 15673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 07:10:58.466059 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15555
2026-08-08T07:10:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1463 ms)
[----------] 18 tests from AuthTokenIssuingTest (27400 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:10:58 dist-test-slave-bw88 krb5kdc[15698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:10:58 dist-test-slave-bw88 krb5kdc[15698](info): set up 2 sockets
Aug 08 07:10:58 dist-test-slave-bw88 krb5kdc[15698](info): commencing operation
krb5kdc: starting...
W20260808 07:10:59.756158 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:10:59 dist-test-slave-bw88 krb5kdc[15698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:10:59.881474 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:32817
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:10:59.967612 15719 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:10:59.967841 15719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:10:59.967872 15719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:10:59.967913 15719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:10:59.969034 15719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:10:59.969096 15719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:10:59.969123 15719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:10:59.969154 15719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:10:59.969180 15719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:10:59.970711 15719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:10:59.970985 15719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:10:59.971228 15719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:10:59.976231 15732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:10:59.976260 15726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:59.976358 15719 server_base.cc:1034] running on GCE node
W20260808 07:10:59.976449 15734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:10:59.976722 15719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:10:59.976960 15719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:10:59.978098 15719 hybrid_clock.cc:648] HybridClock initialized: now 1786173059978047 us; error 63 us; skew 500 ppm
Aug 08 07:10:59 dist-test-slave-bw88 krb5kdc[15698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:10:59.980533 15719 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:10:59.980911 15719 webserver.cc:460] Webserver started at http://127.18.63.126:37581/ using document root <none> and password file <none>
I20260808 07:10:59.981113 15719 fs_manager.cc:359] Metadata directory not provided
I20260808 07:10:59.981155 15719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:10:59.981259 15719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:10:59.982342 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "7cb772d0c11246a28ef68fb2059dde79"
format_stamp: "Formatted at 2026-08-08 07:10:59 on dist-test-slave-bw88"
server_key: "ec1c3811b6754e09426e5a288d12bfe7"
server_key_iv: "7d4776c389d1e374d615d09dd1bff91b"
server_key_version: "encryptionkey@0"
I20260808 07:10:59.982643 15719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "7cb772d0c11246a28ef68fb2059dde79"
format_stamp: "Formatted at 2026-08-08 07:10:59 on dist-test-slave-bw88"
server_key: "ec1c3811b6754e09426e5a288d12bfe7"
server_key_iv: "7d4776c389d1e374d615d09dd1bff91b"
server_key_version: "encryptionkey@0"
I20260808 07:10:59.983978 15719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:10:59.984761 15744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:10:59.984949 15719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 07:10:59.985069 15719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "7cb772d0c11246a28ef68fb2059dde79"
format_stamp: "Formatted at 2026-08-08 07:10:59 on dist-test-slave-bw88"
server_key: "ec1c3811b6754e09426e5a288d12bfe7"
server_key_iv: "7d4776c389d1e374d615d09dd1bff91b"
server_key_version: "encryptionkey@0"
I20260808 07:10:59.985164 15719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:00.016165 15719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:00.018507 15719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:00.018692 15719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:00.023213 15719 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:32817
I20260808 07:11:00.023257 15816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:32817 every 8 connection(s)
I20260808 07:11:00.023650 15719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:00.024753 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:00.026546 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:00.027356 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15719
I20260808 07:11:00.027506 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:11:00.027741 18685 external_mini_cluster.cc:1428] Setting key c636123b9c5f642368447002a73895cd
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.63.125@KRBTEST.COM; defaulting to no policy
I20260808 07:11:00.038089 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.027272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:32817 (local address 127.0.0.1:42408)
0808 07:11:00.027536 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.027588 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.028390 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.028733 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.028736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.029197 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.029488 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.029496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.030615 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.030616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.031306 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.031308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.031849 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.031850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.031916 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.032542 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.032551 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.034107 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.036469 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.036476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.036478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.036674 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.036978 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.036979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.036979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.037072 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.037608 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.037614 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.037673 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.037751 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.037878 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":129,"threads_started":1}
I20260808 07:11:00.038174 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.027367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.18.63.126:32817)
0808 07:11:00.027557 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.027568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.027713 (+ 145us) server_negotiation.cc:407] Connection header received
0808 07:11:00.028583 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.028585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.028602 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.028684 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.029623 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.030481 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.031412 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.031728 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.031856 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.034205 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.034215 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.034215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.034224 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.036363 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.036763 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.036767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.036768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.036891 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.037147 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.037148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.037148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.037530 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.037631 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.037821 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.037979 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260808 07:11:00.038985 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:00.039387 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.039268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57652)
0808 07:11:00.039318 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.039361 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 07:11:00.039546 15746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
I20260808 07:11:00.040022 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.039934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57660)
0808 07:11:00.039976 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.040006 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:11:00.040148 15817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:40239: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
I20260808 07:11:00.060214 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:40239
--webserver_interface=127.18.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:00.089382 15817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } attempt: 1
I20260808 07:11:00.089957 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.089791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57672)
0808 07:11:00.089861 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.089915 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:11:00.090385 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.090276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57682)
0808 07:11:00.090325 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.090358 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:11:00.090565 15817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:40239: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
W20260808 07:11:00.146420 15827 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:11:00.146620 15827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:00.146642 15827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:00.146661 15827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:00.147847 15827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:00.147891 15827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:00.147917 15827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:00.147929 15827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:00.147980 15827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:00.149466 15827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:00.149724 15827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:00.149948 15827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:00.155007 15840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:00.155025 15835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:00.155037 15836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:00.155231 15827 server_base.cc:1034] running on GCE node
I20260808 07:11:00.155458 15827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:00.155767 15827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:00.156924 15827 hybrid_clock.cc:648] HybridClock initialized: now 1786173060156892 us; error 50 us; skew 500 ppm
I20260808 07:11:00.158751 15817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } attempt: 2
I20260808 07:11:00.159197 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.159043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57692)
0808 07:11:00.159118 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.159158 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:00.159612 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.159491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57704)
0808 07:11:00.159550 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.159583 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:11:00.159722 15817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:40239: Network error: Client connection negotiation failed: client connection to 127.18.63.125:40239: connect: Connection refused (error 111)
I20260808 07:11:00.159741 15827 init.cc:377] Logged in from keytab as kudu/127.18.63.125@KRBTEST.COM (short username kudu)
I20260808 07:11:00.160158 15827 webserver.cc:460] Webserver started at http://127.18.63.125:46471/ using document root <none> and password file <none>
I20260808 07:11:00.160367 15827 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:00.160408 15827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:00.160499 15827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:00.161615 15827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data/instance:
uuid: "8371be826057491b9b7ae439900ca9e4"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "599cf618ff3954347d50715f3e1aba62"
server_key_iv: "6554aae96a88a56a1da398c356bbaf87"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.161929 15827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal/instance:
uuid: "8371be826057491b9b7ae439900ca9e4"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "599cf618ff3954347d50715f3e1aba62"
server_key_iv: "6554aae96a88a56a1da398c356bbaf87"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.163240 15827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:00.164019 15850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:00.164227 15827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:00.164314 15827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "8371be826057491b9b7ae439900ca9e4"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "599cf618ff3954347d50715f3e1aba62"
server_key_iv: "6554aae96a88a56a1da398c356bbaf87"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.164367 15827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:00.175263 15827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:00.177567 15827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:00.177686 15827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:00.181842 15827 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.125:40239
I20260808 07:11:00.181852 15924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.125:40239 every 8 connection(s)
I20260808 07:11:00.182250 15827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
I20260808 07:11:00.183301 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:00.184863 15926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:00.186261 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15827
I20260808 07:11:00.186383 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-1/wal/instance
I20260808 07:11:00.186563 18685 external_mini_cluster.cc:1428] Setting key 73b6dc32d5137e1e577a5b7514309048
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.63.124@KRBTEST.COM; defaulting to no policy
I20260808 07:11:00.195197 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.185587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.18.63.126:32817)
0808 07:11:00.185658 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.185661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.185948 (+ 287us) server_negotiation.cc:407] Connection header received
0808 07:11:00.186677 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.186679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.186695 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.186772 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.187802 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.188306 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.189575 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.189799 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.189881 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.192097 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.192100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.192100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.192108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.193738 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.194181 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.194182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.194183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.194250 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.194528 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.194529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.194529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.194651 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.194729 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.195012 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.195102 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 07:11:00.195399 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.185578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:32817 (local address 127.0.0.1:42412)
0808 07:11:00.185805 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.185842 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.186557 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.186849 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.186853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.187277 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.187634 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.187640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.188464 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.188466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.189469 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.189472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.189903 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.189905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.190042 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.190560 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.190569 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.191997 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.193830 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.193838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.193840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.194094 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.194331 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.194332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.194333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.194444 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.194722 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.194730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.194874 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.194997 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.195159 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":113,"threads_started":1}
I20260808 07:11:00.195942 15926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.124@KRBTEST.COM" created.
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:00.204033 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.196220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57710)
0808 07:11:00.196271 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.196281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.196321 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.196632 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.196633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.196774 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.196832 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.196833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.197785 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.197786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.198278 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.198280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.198781 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.198782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.198847 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.199284 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.199289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.200699 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.202908 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.202909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.202909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.203113 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.203376 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.203376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.203377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.203434 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.203794 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.203794 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.203834 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.203887 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.203964 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:11:00.204229 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.196332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.18.63.125:40239)
0808 07:11:00.196481 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.196486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.196510 (+ 24us) server_negotiation.cc:407] Connection header received
0808 07:11:00.196532 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.196533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.196544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.196632 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.196918 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.197686 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.198376 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.198699 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.198785 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.200789 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.200798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.200798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.200806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.202817 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.203224 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.203228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.203228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.203299 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.203513 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.203513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.203514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.203719 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.203814 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.203926 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.204034 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260808 07:11:00.205003 15926 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:00.205523 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.205383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60614)
0808 07:11:00.205427 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.205491 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 07:11:00.205684 15851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
I20260808 07:11:00.206189 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.206074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60628)
0808 07:11:00.206130 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.206161 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:11:00.206386 15926 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:39825: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
I20260808 07:11:00.217290 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:39825
--webserver_interface=127.18.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:00.250895 15817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } attempt: 3
I20260808 07:11:00.255656 15926 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } attempt: 1
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:00.256143 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.255979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60636)
0808 07:11:00.256051 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.256095 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:11:00.256592 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.256428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60640)
0808 07:11:00.256508 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.256550 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 07:11:00.256852 15926 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:39825: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
I20260808 07:11:00.259879 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.251481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.18.63.125:40239)
0808 07:11:00.251544 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.251547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.251566 (+ 19us) server_negotiation.cc:407] Connection header received
0808 07:11:00.251691 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.251692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.251706 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.251770 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.252142 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.252950 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.253876 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.254218 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.254297 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.256685 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.256687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.256688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.256696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.258514 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.258951 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.258952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.258953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.259049 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.259287 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.259288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.259289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.259422 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.259514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.259691 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.259780 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 07:11:00.259886 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.251390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57724)
0808 07:11:00.251488 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.251505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.251581 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.251797 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.251798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.251965 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.252054 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.252056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.253050 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.253051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.253788 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.253791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.254303 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.254304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.254376 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.254917 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.254923 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.256576 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.258634 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.258635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.258636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.258838 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.259132 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.259133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.259134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.259206 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.259506 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.259507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.259561 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.259661 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.259780 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 07:11:00.260537 15817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:00.260900 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.260799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60644)
0808 07:11:00.260845 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.260879 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 07:11:00.261266 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.261154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60660)
0808 07:11:00.261217 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.261244 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:11:00.261391 15817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:39825: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
I20260808 07:11:00.292533 15817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } attempt: 1
I20260808 07:11:00.293057 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.292904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60672)
0808 07:11:00.292980 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.293023 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 07:11:00.293557 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.293447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60676)
0808 07:11:00.293498 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.293534 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:11:00.293661 15817 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:39825: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
W20260808 07:11:00.305303 15938 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:11:00.305636 15938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:00.305676 15938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:00.305722 15938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:00.306880 15938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:00.306944 15938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:00.306967 15938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:00.307004 15938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:00.307036 15938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:00.308591 15938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:32817,127.18.63.125:40239,127.18.63.124:39825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:39825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:00.308876 15938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:00.309144 15938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:00.314229 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:00.314275 15952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:00.314306 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:00.314812 15938 server_base.cc:1034] running on GCE node
I20260808 07:11:00.315104 15938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:00.315486 15938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:00.316745 15938 hybrid_clock.cc:648] HybridClock initialized: now 1786173060316719 us; error 45 us; skew 500 ppm
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:00.319648 15938 init.cc:377] Logged in from keytab as kudu/127.18.63.124@KRBTEST.COM (short username kudu)
I20260808 07:11:00.320204 15938 webserver.cc:460] Webserver started at http://127.18.63.124:45283/ using document root <none> and password file <none>
I20260808 07:11:00.320465 15938 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:00.320514 15938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:00.320664 15938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:00.321831 15938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data/instance:
uuid: "3ac59ceed7674280870dccd02edd0b39"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "2b7f95621ef3620092855bd2f7aa577e"
server_key_iv: "f4a8dee507d98db73d07534111e2ce54"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.322211 15938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal/instance:
uuid: "3ac59ceed7674280870dccd02edd0b39"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "2b7f95621ef3620092855bd2f7aa577e"
server_key_iv: "f4a8dee507d98db73d07534111e2ce54"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.323603 15938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:00.324380 15962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:00.324599 15938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:00.324693 15938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "3ac59ceed7674280870dccd02edd0b39"
format_stamp: "Formatted at 2026-08-08 07:11:00 on dist-test-slave-bw88"
server_key: "2b7f95621ef3620092855bd2f7aa577e"
server_key_iv: "f4a8dee507d98db73d07534111e2ce54"
server_key_version: "encryptionkey@0"
I20260808 07:11:00.324771 15938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:00.325016 15926 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } attempt: 2
I20260808 07:11:00.325559 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.325269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60680)
0808 07:11:00.325468 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.325526 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 07:11:00.325966 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.325875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60692)
0808 07:11:00.325919 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.325949 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 07:11:00.326077 15926 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:39825: Network error: Client connection negotiation failed: client connection to 127.18.63.124:39825: connect: Connection refused (error 111)
I20260808 07:11:00.335659 15938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:00.337636 15938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:00.337792 15938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:00.342692 15938 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.124:39825
I20260808 07:11:00.342792 16026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.124:39825 every 8 connection(s)
I20260808 07:11:00.343088 15938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
I20260808 07:11:00.344135 16027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:00.345784 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:11:00.353192 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 15938
I20260808 07:11:00.353364 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786172779312146-18685-0/minicluster-data/master-2/wal/instance
I20260808 07:11:00.353600 18685 external_mini_cluster.cc:1428] Setting key 0155bf4834d9482ab8af71f8dd807d54
I20260808 07:11:00.355140 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.346515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.18.63.126:32817)
0808 07:11:00.346624 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.346627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.346789 (+ 162us) server_negotiation.cc:407] Connection header received
0808 07:11:00.347311 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.347313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.347326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.347378 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.348070 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.348553 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.349676 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.349882 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.349977 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.352024 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.352026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.352026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.352034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.353638 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.354102 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.354104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.354105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.354179 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.354447 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.354448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.354448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.354583 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.354659 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.354955 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.355047 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 07:11:00.355335 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.346430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:32817 (local address 127.0.0.1:42428)
0808 07:11:00.346667 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.346703 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.347190 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.347447 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.347450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.347768 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.347967 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.347972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.348653 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.348654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.349577 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.349580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.350008 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.350010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.350155 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.350595 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.350603 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.351920 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.353748 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.353755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.353757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.354011 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.354249 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.354251 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.354251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.354373 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.354662 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.354670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.354814 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.354917 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.355095 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260808 07:11:00.355921 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:00.363111 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.356207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:40239 (local address 127.0.0.1:57736)
0808 07:11:00.356261 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.356272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.356306 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.356440 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.356440 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.356598 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.356648 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.356650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.357319 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.357321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.357905 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.357908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.358486 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.358486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.358550 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.358931 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.358935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.360466 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.362062 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.362064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.362237 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.362454 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.362455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.362527 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.362784 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.362785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.362829 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.362913 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.363010 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":11}
I20260808 07:11:00.363126 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.356270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.18.63.125:40239)
0808 07:11:00.356314 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.356315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.356326 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:00.356356 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.356362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.356371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.356420 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.356733 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.357216 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.358055 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.358399 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.358490 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.360572 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.360573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.360573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.360581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.361943 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362317 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.362318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.362319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.362382 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362599 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.362600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.362600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.362713 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.362792 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.362956 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.363039 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 07:11:00.363402 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.354033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.18.63.124:39825)
0808 07:11:00.354253 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.354261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.354301 (+ 40us) server_negotiation.cc:407] Connection header received
0808 07:11:00.354374 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.354376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.354392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.354479 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.355084 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.356075 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.357085 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.357490 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.357576 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.360095 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.360103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.360103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.360109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.361967 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362388 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.362391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.362391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.362453 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.362675 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.362675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.362676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.362830 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.362983 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.363062 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.363146 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
I20260808 07:11:00.363973 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:11:00.368803 15817 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } attempt: 2
I20260808 07:11:00.371734 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.364278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60712)
0808 07:11:00.364354 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.364362 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.364402 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.364542 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.364543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.364693 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.364731 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.364733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.365571 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.365571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.366233 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.366236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.366668 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.366669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.366708 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.367157 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.367164 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.368359 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.370536 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.370538 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.370538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.370799 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.371021 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.371021 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.371022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.371086 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.371389 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.371390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.371428 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.371493 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.371601 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 07:11:00.371742 16033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.364322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.18.63.124:39825)
0808 07:11:00.364360 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.364362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.364391 (+ 29us) server_negotiation.cc:407] Connection header received
0808 07:11:00.364451 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.364452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.364463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.364537 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.364824 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.365449 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.366324 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.366582 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.366654 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.368485 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.368486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.368487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.368493 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.370436 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.370883 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.370884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.370884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.370947 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.371159 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.371160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.371160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.371301 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.371365 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.371535 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.371649 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":17}
I20260808 07:11:00.372252 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.364328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.18.63.126:32817)
0808 07:11:00.364369 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.364370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.364379 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:00.364455 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.364456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.364465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.364527 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.365104 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.365745 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.366596 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.366826 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.366907 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.369067 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.369070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.369070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.369078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.370862 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.371282 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.371283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.371284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.371340 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.371576 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.371577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.371577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.371702 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.371851 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.372065 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.372164 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:11:00.374171 16027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: Bootstrap starting.
I20260808 07:11:00.374949 16027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:00.375355 16027 log.cc:824] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:00.376370 16027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: No bootstrap required, opened a new log
I20260808 07:11:00.377468 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.369055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60724)
0808 07:11:00.369099 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.369111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.369149 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.369483 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.369484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.369638 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.369670 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.369672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.371220 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.371222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.371989 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.371992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.372545 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.372546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.372600 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.373041 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.373045 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.374400 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.376303 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.376304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.376305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.376537 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.376784 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.376785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.376785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.376843 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.377150 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.377150 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.377193 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.377240 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.377362 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:00.377869 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.369139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.18.63.124:39825)
0808 07:11:00.369351 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.369353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.369361 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:00.369379 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.369380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.369389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.369477 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.369755 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.371161 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.372076 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.372437 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.372741 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.374498 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.374499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.374500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.374507 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.376199 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.376631 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.376632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.376633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.376708 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.376911 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.376912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.376912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.377064 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.377602 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.377645 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.377729 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":143,"threads_started":1}
I20260808 07:11:00.378731 16027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.378903 16027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac59ceed7674280870dccd02edd0b39, State: Initialized, Role: FOLLOWER
I20260808 07:11:00.379002 16027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.379392 16027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:00.379346 16040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.379590 16040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.380153 15817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: Bootstrap starting.
I20260808 07:11:00.380932 15817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:00.381306 15817 log.cc:824] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: Log is configured to *not* fsync() on all Append() calls
W20260808 07:11:00.381804 16060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:00.381860 16060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:00.382072 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.372978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.18.63.125:40239)
0808 07:11:00.373028 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.373031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.373489 (+ 458us) server_negotiation.cc:407] Connection header received
0808 07:11:00.373609 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.373610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.373621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.373690 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.374166 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.374752 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.375546 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.375897 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.375970 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.378377 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.378378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.378379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.378386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.380253 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.380669 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.380670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.380670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.380730 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.380946 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.380948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.380948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.381090 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.381281 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.381882 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.381977 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 07:11:00.383651 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:00.381901 15938 master_runner.cc:186] Error getting master registration for 127.18.63.126:32817: OK, instance_id { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" instance_seqno: 1786173060021935 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:11:00.384330 15938 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:11:00.385087 15817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: No bootstrap required, opened a new log
I20260808 07:11:00.384667 15784 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7cb772d0c11246a28ef68fb2059dde79"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42434
I20260808 07:11:00.386762 15817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.386896 15817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb772d0c11246a28ef68fb2059dde79, State: Initialized, Role: FOLLOWER
I20260808 07:11:00.386989 15817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.387318 15817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:00.387378 15784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:11:00.387465 15784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:00.388307 15784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.388564 15784 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [CANDIDATE]: Term 1 election: Requested vote from peers 8371be826057491b9b7ae439900ca9e4 (127.18.63.125:40239), 3ac59ceed7674280870dccd02edd0b39 (127.18.63.124:39825)
I20260808 07:11:00.389071 15995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb772d0c11246a28ef68fb2059dde79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ac59ceed7674280870dccd02edd0b39"
I20260808 07:11:00.389189 15995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:00.390573 15995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cb772d0c11246a28ef68fb2059dde79 in term 1.
I20260808 07:11:00.390797 15745 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac59ceed7674280870dccd02edd0b39, 7cb772d0c11246a28ef68fb2059dde79; no voters:
W20260808 07:11:00.391673 16077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:00.391726 16077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:00.391804 16085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:00.391933 16085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb772d0c11246a28ef68fb2059dde79, State: Running, Role: LEADER
I20260808 07:11:00.392021 16085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.392400 16085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb772d0c11246a28ef68fb2059dde79. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.392496 16085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:00.391337 15719 master_runner.cc:186] Error getting master registration for 127.18.63.125:40239: OK, instance_id { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" instance_seqno: 1786173060180532 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:11:00.393651 15719 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:11:00.387322 16063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.394318 16063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.401474 16091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:00.401738 16091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:00.404214 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.394695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.18.63.125:40239)
0808 07:11:00.394731 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.394733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.396117 (+ 1384us) server_negotiation.cc:407] Connection header received
0808 07:11:00.396607 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.396608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.396619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.396648 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.397282 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.398054 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.399105 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.399409 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.399448 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.400949 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.400951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.400951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.400959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.402887 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.403311 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.403312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.403313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.403376 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.403589 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.403590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.403591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.403719 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.403807 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.404017 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.404114 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:00.404328 15995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [term 1 FOLLOWER]: Refusing update from remote peer 7cb772d0c11246a28ef68fb2059dde79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:00.404765 16063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:00.406185 16040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb772d0c11246a28ef68fb2059dde79. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.406286 16040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.406567 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.393948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.18.63.124:39825)
0808 07:11:00.393984 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.393986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.394813 (+ 827us) server_negotiation.cc:407] Connection header received
0808 07:11:00.394966 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.394967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.394989 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.395018 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.396053 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.397661 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.399717 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.400094 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.400156 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.402577 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.402579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.402579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.402587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.404434 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.405289 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.405290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.405291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.405371 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.405610 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.405611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.405611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.405759 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.405856 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.406260 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.406459 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 07:11:00.407943 16063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.408046 16063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:00.408207 16097 mvcc.cc:205] Tried to move back new op lower bound from 7316164855412170752 to 7316164855366221824. Current Snapshot: MvccSnapshot[applied={T|T < 7316164855412170752}]
I20260808 07:11:00.408545 16040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.408618 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.390265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.18.63.126:32817)
0808 07:11:00.390373 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.390376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.390543 (+ 167us) server_negotiation.cc:407] Connection header received
0808 07:11:00.390721 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.390722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.390734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.390767 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.391702 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.392398 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.393694 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.393970 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.394015 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.396239 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.396240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.396240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.396247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.398643 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.407572 (+ 8929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.407573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.407574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.407642 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.407857 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.407858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.407859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.407996 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.408069 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.408435 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.408531 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 07:11:00.408624 16040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.408937 16102 mvcc.cc:205] Tried to move back new op lower bound from 7316164855412170752 to 7316164855366221824. Current Snapshot: MvccSnapshot[applied={T|T < 7316164855412170752}]
I20260808 07:11:00.413739 16091 catalog_manager.cc:1347] Generated new cluster ID: a13c9b28b91742be8d459ade8faf7452
I20260808 07:11:00.413796 16091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:00.417258 15926 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } attempt: 3
Aug 08 07:11:00 dist-test-slave-bw88 krb5kdc[15698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173060, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:11:00.426721 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.417548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:39825 (local address 127.0.0.1:60744)
0808 07:11:00.417639 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:00.417650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:00.417690 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:00.418436 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:00.418437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:00.418633 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:00.418688 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.418690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.419666 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.419667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.420481 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:00.420484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.420999 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.421000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:00.421059 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.421638 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:00.421643 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:00.423751 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:00.425674 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.425675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.425675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.425885 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.426092 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:00.426092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:00.426093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:00.426149 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:00.426413 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:00.426414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:00.426461 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 07:11:00.426508 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 07:11:00.426618 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 07:11:00.427112 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:00.417767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.18.63.124:39825)
0808 07:11:00.418307 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.418309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.418318 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:00.418340 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.418341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.418350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.418759 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.418792 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.419564 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.420601 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:00.420916 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:00.421260 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:00.423843 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:00.423844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:00.423845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:00.423851 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:00.425575 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.425966 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.425967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.425968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.426020 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:00.426217 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:00.426217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:00.426218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:00.426342 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:00.426900 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:00.426946 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:11:00.427020 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519}
I20260808 07:11:00.428764 15926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: Bootstrap starting.
I20260808 07:11:00.429596 15926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:00.429960 15926 log.cc:824] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:00.431177 15926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: No bootstrap required, opened a new log
I20260808 07:11:00.433296 15926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.433468 15926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8371be826057491b9b7ae439900ca9e4, State: Initialized, Role: FOLLOWER
I20260808 07:11:00.433566 15926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } }
I20260808 07:11:00.433878 15926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:00.433914 15888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cb772d0c11246a28ef68fb2059dde79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8371be826057491b9b7ae439900ca9e4"
I20260808 07:11:00.434032 15888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:00.434875 15888 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cb772d0c11246a28ef68fb2059dde79 in term 1.
I20260808 07:11:00.435077 15887 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [term 1 FOLLOWER]: Refusing update from remote peer 7cb772d0c11246a28ef68fb2059dde79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:00.435696 16088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:00.436183 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.436268 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.437793 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb772d0c11246a28ef68fb2059dde79. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.437873 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:00.437906 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.437947 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:11:00.441962 16124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:00.442023 16124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:00.442430 16127 mvcc.cc:205] Tried to move back new op lower bound from 7316164855412170752 to 7316164855366221824. Current Snapshot: MvccSnapshot[applied={T|T < 7316164855412170752}]
I20260808 07:11:00.444437 16063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7cb772d0c11246a28ef68fb2059dde79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb772d0c11246a28ef68fb2059dde79" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 32817 } } peers { permanent_uuid: "8371be826057491b9b7ae439900ca9e4" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 40239 } } peers { permanent_uuid: "3ac59ceed7674280870dccd02edd0b39" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 39825 } } }
I20260808 07:11:00.444559 16063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79 [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:00.446434 16091 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:00.447201 16091 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:00.460857 16091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb772d0c11246a28ef68fb2059dde79: Generated new TSK 0
I20260808 07:11:00.461143 16091 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:11:00.494215 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.492369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57756 (local address 127.18.63.125:40239)
0808 07:11:00.492409 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.492410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.493139 (+ 729us) server_negotiation.cc:407] Connection header received
0808 07:11:00.493587 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.493589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.493601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.493634 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.494140 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57756: BlockingRecv error: recv got EOF from 127.0.0.1:57756 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:11:00.494865 16035 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.492110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.18.63.124:39825)
0808 07:11:00.492161 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.492163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.493248 (+ 1085us) server_negotiation.cc:407] Connection header received
0808 07:11:00.494400 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.494402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.494414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.494442 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.494807 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60748: BlockingRecv error: recv got EOF from 127.0.0.1:60748 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:11:00.495568 15821 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.492593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.18.63.126:32817)
0808 07:11:00.492632 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.492633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.493470 (+ 837us) server_negotiation.cc:407] Connection header received
0808 07:11:00.495102 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.495103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.495113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.495143 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.495504 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42462: BlockingRecv error: recv got EOF from 127.0.0.1:42462 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 07:11:00.498303 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.497470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.18.63.125:40239)
0808 07:11:00.497507 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.497509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.497750 (+ 241us) server_negotiation.cc:407] Connection header received
0808 07:11:00.497861 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.497862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.497881 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.497915 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.498254 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57760: BlockingRecv error: recv got EOF from 127.0.0.1:57760 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:11:00.501377 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.500592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57766 (local address 127.18.63.125:40239)
0808 07:11:00.500628 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.500629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.500839 (+ 210us) server_negotiation.cc:407] Connection header received
0808 07:11:00.500931 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.500932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.500942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.500969 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.501305 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57766: BlockingRecv error: recv got EOF from 127.0.0.1:57766 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:11:00.505390 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.504523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.18.63.125:40239)
0808 07:11:00.504557 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.504559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.504828 (+ 269us) server_negotiation.cc:407] Connection header received
0808 07:11:00.504927 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.504928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.504938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.504968 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.505301 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57768: BlockingRecv error: recv got EOF from 127.0.0.1:57768 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:11:00.511363 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.510515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.18.63.125:40239)
0808 07:11:00.510565 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.510567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.510832 (+ 265us) server_negotiation.cc:407] Connection header received
0808 07:11:00.510934 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.510935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.510945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.510975 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.511320 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57782: BlockingRecv error: recv got EOF from 127.0.0.1:57782 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 07:11:00.521607 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.520655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.18.63.125:40239)
0808 07:11:00.520693 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.520695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.520969 (+ 274us) server_negotiation.cc:407] Connection header received
0808 07:11:00.521090 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.521091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.521102 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.521131 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.521545 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57786: BlockingRecv error: recv got EOF from 127.0.0.1:57786 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:11:00.540062 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.539040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57794 (local address 127.18.63.125:40239)
0808 07:11:00.539086 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.539088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.539384 (+ 296us) server_negotiation.cc:407] Connection header received
0808 07:11:00.539500 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.539501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.539516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.539548 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.539973 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57794: BlockingRecv error: recv got EOF from 127.0.0.1:57794 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 07:11:00.574329 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.573537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.18.63.125:40239)
0808 07:11:00.573587 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.573590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.573751 (+ 161us) server_negotiation.cc:407] Connection header received
0808 07:11:00.573831 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.573832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.573844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.573902 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.574258 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57810: BlockingRecv error: recv got EOF from 127.0.0.1:57810 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 07:11:00.640369 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.639592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.18.63.125:40239)
0808 07:11:00.639665 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.639667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.639761 (+ 94us) server_negotiation.cc:407] Connection header received
0808 07:11:00.639866 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.639868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.639880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.639943 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.640299 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57814: BlockingRecv error: recv got EOF from 127.0.0.1:57814 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 07:11:00.770732 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:00.769853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.18.63.125:40239)
0808 07:11:00.769921 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:00.769924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:00.770066 (+ 142us) server_negotiation.cc:407] Connection header received
0808 07:11:00.770177 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:00.770178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:00.770197 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:00.770262 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:00.770683 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57816: BlockingRecv error: recv got EOF from 127.0.0.1:57816 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 07:11:01.029258 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:01.028433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.18.63.125:40239)
0808 07:11:01.028511 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.028518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.028625 (+ 107us) server_negotiation.cc:407] Connection header received
0808 07:11:01.028731 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.028732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.028750 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.028818 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:01.029210 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57826: BlockingRecv error: recv got EOF from 127.0.0.1:57826 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 07:11:01.382344 16060 catalog_manager.cc:1259] Loaded cluster ID: a13c9b28b91742be8d459ade8faf7452
I20260808 07:11:01.382454 16060 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: loading cluster ID for follower catalog manager: success
I20260808 07:11:01.383332 16060 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: acquiring CA information for follower catalog manager: success
I20260808 07:11:01.383671 16060 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ac59ceed7674280870dccd02edd0b39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:11:01.442516 16124 catalog_manager.cc:1259] Loaded cluster ID: a13c9b28b91742be8d459ade8faf7452
I20260808 07:11:01.442721 16124 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: loading cluster ID for follower catalog manager: success
I20260808 07:11:01.443567 16124 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: acquiring CA information for follower catalog manager: success
I20260808 07:11:01.443897 16124 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8371be826057491b9b7ae439900ca9e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:11:01.545161 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.542674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.18.63.125:40239)
0808 07:11:01.542850 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.542852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.542880 (+ 28us) server_negotiation.cc:407] Connection header received
0808 07:11:01.542938 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.542938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.542998 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.543080 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:01.543237 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.543732 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.544475 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.544672 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.544725 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:01.544867 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:01.544978 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:01.545042 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:01.545071 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 07:11:01.545090 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260808 07:11:01.549340 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.547051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.18.63.124:39825)
0808 07:11:01.547177 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.547180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.547188 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:01.547227 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.547228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.547266 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.547336 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:01.547445 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.547951 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.548652 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.548840 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.548892 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:01.549022 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:01.549117 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:01.549171 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:01.549211 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:11:01.549227 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260808 07:11:01.554425 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.551186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.18.63.124:39825)
0808 07:11:01.551226 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.551228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.551541 (+ 313us) server_negotiation.cc:407] Connection header received
0808 07:11:01.551591 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.551592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.551603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.551650 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:01.551807 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.552513 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.553654 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.553950 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.554022 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:01.554175 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:01.554223 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:01.554277 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:01.554323 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 07:11:01.554344 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:11:01.554924 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.551144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57848 (local address 127.18.63.125:40239)
0808 07:11:01.551206 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.551208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.551390 (+ 182us) server_negotiation.cc:407] Connection header received
0808 07:11:01.551496 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.551498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.551509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.551574 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:01.551728 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.552455 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.554068 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.554368 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.554435 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:01.554633 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:01.554673 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:01.554740 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:01.554803 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 07:11:01.554826 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 07:11:01.555332 16240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:01.551208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.18.63.126:32817)
0808 07:11:01.551343 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:01.551346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:01.551429 (+ 83us) server_negotiation.cc:407] Connection header received
0808 07:11:01.551476 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:01.551477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:01.551546 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:01.551626 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:01.552615 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.553205 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.554256 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:01.554584 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:01.554654 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:01.554879 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:01.554986 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:01.555067 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:01.555116 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 07:11:01.555141 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260808 07:11:01.557267 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15719
I20260808 07:11:01.571908 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15827
I20260808 07:11:01.579869 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 15938
2026-08-08T07:11:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3115 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 07:11:01 dist-test-slave-bw88 krb5kdc[16245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 07:11:01 dist-test-slave-bw88 krb5kdc[16245](info): set up 2 sockets
Aug 08 07:11:01 dist-test-slave-bw88 krb5kdc[16245](info): commencing operation
krb5kdc: starting...
W20260808 07:11:02.778805 18685 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/test-user.keytab.
Aug 08 07:11:02 dist-test-slave-bw88 krb5kdc[16245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T07:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T07:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
Entry for principal kudu/127.18.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab.
I20260808 07:11:02.904345 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44283
--webserver_interface=127.18.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 07:11:02.987790 16263 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:11:02.988011 16263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:02.988029 16263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:02.988049 16263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:02.989135 16263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:02.989166 16263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:02.989180 16263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:02.989192 16263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:02.989204 16263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:02.990641 16263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.126.keytab
--principal=kudu/127.18.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.126:44283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:02.990844 16263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:02.991148 16263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:02.996516 16278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:02.996564 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:02.996573 16275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:02.996862 16263 server_base.cc:1034] running on GCE node
I20260808 07:11:02.997153 16263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:02.997509 16263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:02.998704 16263 hybrid_clock.cc:648] HybridClock initialized: now 1786173062998678 us; error 44 us; skew 500 ppm
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:03.005431 16263 init.cc:377] Logged in from keytab as kudu/127.18.63.126@KRBTEST.COM (short username kudu)
I20260808 07:11:03.005908 16263 webserver.cc:460] Webserver started at http://127.18.63.126:36045/ using document root <none> and password file <none>
I20260808 07:11:03.006157 16263 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:03.006202 16263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:03.006299 16263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:03.007373 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data/instance:
uuid: "675a64b57a4c4875bb20e1feabf47adb"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "79cabce6707a6214278397334e6faab4"
server_key_iv: "c7bc93194101deba54ee287a55d51285"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.007699 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance:
uuid: "675a64b57a4c4875bb20e1feabf47adb"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "79cabce6707a6214278397334e6faab4"
server_key_iv: "c7bc93194101deba54ee287a55d51285"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.009101 16263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 07:11:03.009986 16286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:03.010195 16263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 07:11:03.010293 16263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal
uuid: "675a64b57a4c4875bb20e1feabf47adb"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "79cabce6707a6214278397334e6faab4"
server_key_iv: "c7bc93194101deba54ee287a55d51285"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.010357 16263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:03.034590 16263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:03.036852 16263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:03.036998 16263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:03.041811 16263 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.126:44283
I20260808 07:11:03.041886 16356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.126:44283 every 8 connection(s)
I20260808 07:11:03.042271 16263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/data/info.pb
I20260808 07:11:03.043509 16359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:03.045248 16359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.049816 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 16263
I20260808 07:11:03.049947 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-0/wal/instance
I20260808 07:11:03.050189 18685 external_mini_cluster.cc:1428] Setting key 53e096cc5a50483e0da9bd196445809e
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:11:03.056811 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.046265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.18.63.126:44283)
0808 07:11:03.046437 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.046442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.046478 (+ 36us) server_negotiation.cc:407] Connection header received
0808 07:11:03.046915 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.046917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.046943 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.047033 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.047596 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.048477 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.049960 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.050467 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.050563 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.053409 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.053421 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.053421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.053427 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.055073 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.055627 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.055630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.055630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.055708 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.055978 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.055978 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.055979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.056188 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.056254 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.056526 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.056648 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260808 07:11:03.056950 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.046088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44283 (local address 127.0.0.1:55546)
0808 07:11:03.046279 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.046315 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.046767 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.047063 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.047066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.047323 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.047478 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.047482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.048596 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.048597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.049821 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.049824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.050592 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.050594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.050677 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.051199 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.051210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.053300 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.055211 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.055220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.055222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.055528 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.055786 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.055787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.055787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.055908 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.056275 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.056282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.056367 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.056491 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.056765 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":91,"thread_start_us":86,"threads_started":1}
I20260808 07:11:03.057689 16359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.058023 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.057927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41322)
0808 07:11:03.057962 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.058004 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260808 07:11:03.058192 16288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
I20260808 07:11:03.058650 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.058555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41338)
0808 07:11:03.058607 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.058632 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 07:11:03.058786 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45547: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
Entry for principal kudu/127.18.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab.
I20260808 07:11:03.080856 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45547
--webserver_interface=127.18.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:03.108011 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } attempt: 1
I20260808 07:11:03.108487 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.108352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41350)
0808 07:11:03.108417 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.108457 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 07:11:03.108911 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.108822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41362)
0808 07:11:03.108870 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.108895 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 07:11:03.109021 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45547: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
W20260808 07:11:03.167281 16367 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:11:03.167536 16367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:03.167567 16367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:03.167616 16367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:03.168849 16367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:03.168915 16367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:03.168938 16367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:03.168959 16367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:03.168988 16367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:03.170480 16367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.125.keytab
--principal=kudu/127.18.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.125:45547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:03.170787 16367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:03.171013 16367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 07:11:03.175994 16377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:03.175994 16374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:03.175984 16379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:03.176234 16367 server_base.cc:1034] running on GCE node
I20260808 07:11:03.176465 16367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:03.176709 16367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:03.177140 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } attempt: 2
I20260808 07:11:03.177587 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.177450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41370)
0808 07:11:03.177505 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.177537 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:11:03.177857 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.177798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41372)
0808 07:11:03.177825 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.177845 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 07:11:03.177857 16367 hybrid_clock.cc:648] HybridClock initialized: now 1786173063177842 us; error 29 us; skew 500 ppm
W20260808 07:11:03.177958 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.125:45547: Network error: Client connection negotiation failed: client connection to 127.18.63.125:45547: connect: Connection refused (error 111)
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:03.180370 16367 init.cc:377] Logged in from keytab as kudu/127.18.63.125@KRBTEST.COM (short username kudu)
I20260808 07:11:03.180822 16367 webserver.cc:460] Webserver started at http://127.18.63.125:43361/ using document root <none> and password file <none>
I20260808 07:11:03.181003 16367 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:03.181041 16367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:03.181128 16367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:03.182183 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data/instance:
uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "bbb76b721ad6c04a76eda2cf7f134eb6"
server_key_iv: "ec7fbde1221126a9d17f8557c68edf7f"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.182487 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal/instance:
uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "bbb76b721ad6c04a76eda2cf7f134eb6"
server_key_iv: "ec7fbde1221126a9d17f8557c68edf7f"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.183758 16367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 07:11:03.184407 16388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:03.184607 16367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 07:11:03.184717 16367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal
uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "bbb76b721ad6c04a76eda2cf7f134eb6"
server_key_iv: "ec7fbde1221126a9d17f8557c68edf7f"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.184799 16367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:03.208297 16367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:03.210748 16367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:03.210927 16367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:03.215592 16367 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.125:45547
I20260808 07:11:03.215631 16463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.125:45547 every 8 connection(s)
I20260808 07:11:03.216030 16367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/data/info.pb
I20260808 07:11:03.216285 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 16367
I20260808 07:11:03.216410 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-1/wal/instance
I20260808 07:11:03.216619 18685 external_mini_cluster.cc:1428] Setting key 919d415830fcea605cc788e55539649c
I20260808 07:11:03.217170 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:03.218757 16465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.63.124@KRBTEST.COM" created.
I20260808 07:11:03.229521 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.219501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.18.63.126:44283)
0808 07:11:03.219566 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.219568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.219857 (+ 289us) server_negotiation.cc:407] Connection header received
0808 07:11:03.220621 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.220622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.220632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.220698 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.221659 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.222176 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.223797 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.223998 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.224056 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.226527 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.226529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.226529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.226537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.228028 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.228434 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.228435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.228436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.228498 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.228812 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.228813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.228813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.228933 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.229014 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.229315 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.229427 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 07:11:03.229699 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.219482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44283 (local address 127.0.0.1:55554)
0808 07:11:03.219684 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.219746 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.220519 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.220766 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.220770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.221191 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.221530 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.221538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.222280 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.222281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.223704 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.223708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.224113 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.224115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.224263 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.224801 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.224811 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.226434 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.228119 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.228126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.228129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.228353 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.228587 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.228587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.228588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.228720 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.229006 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.229013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.229161 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.229291 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.229472 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260808 07:11:03.230341 16465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:03.239174 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.230660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41378)
0808 07:11:03.230714 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.230725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.230772 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.231017 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.231018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.231217 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.231280 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.231282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.232757 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.232758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.233243 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.233244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.234123 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.234124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.234249 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.234753 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.234757 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.235873 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.238098 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.238099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.238100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.238257 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.238515 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.238515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.238515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.238587 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.238960 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.238960 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.238995 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.239043 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.239113 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 07:11:03.239303 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.230739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.18.63.125:45547)
0808 07:11:03.230879 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.230883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.230900 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:11:03.230916 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.230918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.230928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.230989 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.231362 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.232643 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.233357 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.233996 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.234106 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.235967 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.235977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.235977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.235986 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.237999 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.238342 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.238346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.238347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.238421 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.238658 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.238660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.238660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.238870 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.238991 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.239086 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.239181 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260808 07:11:03.240121 16465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.240494 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.240359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54256)
0808 07:11:03.240424 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.240475 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 07:11:03.240664 16391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
I20260808 07:11:03.241132 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.241026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54268)
0808 07:11:03.241076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.241099 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:11:03.241299 16465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
Entry for principal kudu/127.18.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab.
I20260808 07:11:03.248360 18685 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskERtyUc/build/release/bin/kudu
/tmp/dist-test-taskERtyUc/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:38155
--webserver_interface=127.18.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.63.84:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 07:11:03.269119 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } attempt: 3
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:03.276528 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.269623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.18.63.125:45547)
0808 07:11:03.269698 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.269700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.269711 (+ 11us) server_negotiation.cc:407] Connection header received
0808 07:11:03.269764 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.269764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.269777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.269851 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.270187 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.270718 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.271425 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.271666 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.271743 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.274020 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.274021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.274022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.274028 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.275429 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.275824 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.275825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.275826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.275879 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.276087 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.276088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.276088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.276194 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.276243 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.276401 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.276462 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 07:11:03.276592 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.269544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41386)
0808 07:11:03.269607 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.269625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.269694 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.269857 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.269858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.270028 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.270104 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.270105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.270815 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.270818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.271339 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.271341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.271755 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.271756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.271837 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.272321 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.272326 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.273912 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.275516 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.275518 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.275518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.275747 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.275953 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.275954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.275955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.276026 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.276255 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.276255 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.276310 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.276387 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.276495 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:11:03.277150 16359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.277520 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.277425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54280)
0808 07:11:03.277464 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.277496 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 07:11:03.277855 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.277796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54292)
0808 07:11:03.277819 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.277841 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260808 07:11:03.277967 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
I20260808 07:11:03.290540 16465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 1
I20260808 07:11:03.290994 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.290859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54302)
0808 07:11:03.290925 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.290966 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 07:11:03.291394 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.291257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54310)
0808 07:11:03.291317 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.291362 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:11:03.291615 16465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
I20260808 07:11:03.309139 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 1
I20260808 07:11:03.309721 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.309567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54326)
0808 07:11:03.309642 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.309689 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 07:11:03.310158 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.310044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54328)
0808 07:11:03.310097 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.310135 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 07:11:03.310268 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
W20260808 07:11:03.338711 16472 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 07:11:03.338973 16472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 07:11:03.339006 16472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 07:11:03.339048 16472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 07:11:03.340245 16472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 07:11:03.340309 16472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 07:11:03.340333 16472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 07:11:03.340353 16472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 07:11:03.340384 16472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 07:11:03.342187 16472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.63.84:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.63.126:44283,127.18.63.125:45547,127.18.63.124:38155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/krb5kdc/kudu_127.18.63.124.keytab
--principal=kudu/127.18.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.63.124:38155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 06:08:33 UTC on ccycloud.quasar-fdzluz.root.comops.site
build id 4574265
I20260808 07:11:03.342536 16472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 07:11:03.342780 16472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 07:11:03.348194 16472 server_base.cc:1034] running on GCE node
W20260808 07:11:03.348150 16483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:03.348140 16478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 07:11:03.348160 16480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 07:11:03.348606 16472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 07:11:03.348912 16472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 07:11:03.350075 16472 hybrid_clock.cc:648] HybridClock initialized: now 1786173063350064 us; error 32 us; skew 500 ppm
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 07:11:03.352669 16472 init.cc:377] Logged in from keytab as kudu/127.18.63.124@KRBTEST.COM (short username kudu)
I20260808 07:11:03.353156 16472 webserver.cc:460] Webserver started at http://127.18.63.124:36431/ using document root <none> and password file <none>
I20260808 07:11:03.353443 16472 fs_manager.cc:359] Metadata directory not provided
I20260808 07:11:03.353494 16472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 07:11:03.353647 16472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 07:11:03.354720 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data/instance:
uuid: "ef31b32414f04d48bcc166d474efdc09"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "f20e101185d1bdb7f30aefb9244ae0a8"
server_key_iv: "fc6e47cae78845c8f5f8536a45acfc3a"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.355100 16472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal/instance:
uuid: "ef31b32414f04d48bcc166d474efdc09"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "f20e101185d1bdb7f30aefb9244ae0a8"
server_key_iv: "fc6e47cae78845c8f5f8536a45acfc3a"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.356396 16472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 07:11:03.357167 16490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 07:11:03.357407 16472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 07:11:03.357554 16472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data,/tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal
uuid: "ef31b32414f04d48bcc166d474efdc09"
format_stamp: "Formatted at 2026-08-08 07:11:03 on dist-test-slave-bw88"
server_key: "f20e101185d1bdb7f30aefb9244ae0a8"
server_key_iv: "fc6e47cae78845c8f5f8536a45acfc3a"
server_key_version: "encryptionkey@0"
I20260808 07:11:03.357646 16472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 07:11:03.359825 16465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 2
I20260808 07:11:03.360285 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.360134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54344)
0808 07:11:03.360201 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.360245 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 07:11:03.360709 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.360592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54346)
0808 07:11:03.360647 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.360680 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:11:03.360821 16465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
I20260808 07:11:03.382622 16472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 07:11:03.384727 16472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 07:11:03.384889 16472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 07:11:03.385424 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 2
I20260808 07:11:03.385836 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.385710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54356)
0808 07:11:03.385771 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.385807 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 07:11:03.386270 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.386130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54364)
0808 07:11:03.386193 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.386238 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 07:11:03.386399 16359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.63.124:38155: Network error: Client connection negotiation failed: client connection to 127.18.63.124:38155: connect: Connection refused (error 111)
I20260808 07:11:03.389633 16472 rpc_server.cc:307] RPC server started. Bound to: 127.18.63.124:38155
I20260808 07:11:03.389664 16572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.63.124:38155 every 8 connection(s)
I20260808 07:11:03.390046 16472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/data/info.pb
I20260808 07:11:03.391050 16573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 07:11:03.392843 16573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.394438 18685 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskERtyUc/build/release/bin/kudu as pid 16472
I20260808 07:11:03.394586 18685 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskERtyUc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786172779312146-18685-0/minicluster-data/master-2/wal/instance
I20260808 07:11:03.394809 18685 external_mini_cluster.cc:1428] Setting key d8243a3baffb979dd920c5930e60ca82
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:11:03.402643 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.393596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.18.63.126:44283)
0808 07:11:03.393670 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.393672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.393845 (+ 173us) server_negotiation.cc:407] Connection header received
0808 07:11:03.394318 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.394319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.394332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.394418 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.395165 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.395646 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.396738 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.396948 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.397044 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.399330 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.399331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.399331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.399337 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.400918 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.401465 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.401466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.401466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.401585 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.401895 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.401896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.401896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.402056 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.402141 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.402440 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.402532 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 07:11:03.402742 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.393528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.126:44283 (local address 127.0.0.1:55570)
0808 07:11:03.393725 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.393764 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.394225 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.394468 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.394471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.394800 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.395057 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.395062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.395749 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.395750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.396638 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.396641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.397082 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.397083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.397191 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.397723 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.397732 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.399233 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.401011 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.401019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.401021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.401395 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.401662 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.401662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.401662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.401826 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.402137 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.402143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.402255 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.402378 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.402546 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260808 07:11:03.403323 16573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.404603 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.395772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.18.63.124:38155)
0808 07:11:03.395939 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.395943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.395960 (+ 17us) server_negotiation.cc:407] Connection header received
0808 07:11:03.395986 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.395988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.396012 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.396115 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.396770 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.397548 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.398306 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.398587 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.398643 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.401410 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.401418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.401418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.401424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.403359 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.403760 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.403763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.403763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.403848 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.404050 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.404051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.404051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.404226 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.404355 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.404451 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.404510 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.126@KRBTEST.COM
I20260808 07:11:03.410583 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.403571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.125:45547 (local address 127.0.0.1:41390)
0808 07:11:03.403622 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.403635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.403681 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.403837 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.403838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.403982 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.404066 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.404068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.404934 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.404935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.405712 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.405715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.406075 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.406076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.406154 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.406570 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.406575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.407857 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.409562 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.409563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.409563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.409780 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.409974 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.409974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.409975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.410044 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.410268 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.410270 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.410313 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.410393 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.410526 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 07:11:03.410605 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.403671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.18.63.125:45547)
0808 07:11:03.403722 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.403725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.403733 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:03.403759 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.403760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.403770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.403824 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.404149 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.404834 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.405790 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.405981 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.406047 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.407936 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.407938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.407938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.407946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.409489 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.409857 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.409858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.409858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.409907 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.410104 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.410104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.410105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.410201 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.410251 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.410413 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.410505 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 07:11:03.411226 16573 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 07:11:03.413421 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.405457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55580 (local address 127.18.63.126:44283)
0808 07:11:03.405496 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.405497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.405506 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:03.405559 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.405559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.405566 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.405606 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.406135 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.406598 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.407417 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.407756 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.407823 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.410476 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.410477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.410478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.410485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.412281 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.412634 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.412635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.412635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.412698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.412900 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.412900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.412901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.412992 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.413103 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.413176 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.413280 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.124@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.63.125@KRBTEST.COM
I20260808 07:11:03.418354 16576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.411450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54388)
0808 07:11:03.411591 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.411601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.411648 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.411780 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.411780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.411885 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.411911 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.411912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.412717 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.412718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.413525 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.413528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.413992 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.413993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.414055 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.414442 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.414447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.415515 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.417279 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.417280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.417280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.417482 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.417717 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.417717 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.417718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.417773 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.418015 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.418015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.418053 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.418146 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.418241 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 07:11:03.418397 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.411521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.18.63.124:38155)
0808 07:11:03.411563 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.411565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.411687 (+ 122us) server_negotiation.cc:407] Connection header received
0808 07:11:03.411708 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.411709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.411720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.411778 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.411978 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.412645 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.413605 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.413900 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.413966 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.415597 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.415598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.415598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.415605 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.417155 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.417568 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.417569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.417569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.417649 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.417842 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.417843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.417844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.417946 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.418022 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.418169 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.418289 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":17}
I20260808 07:11:03.420267 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: Bootstrap starting.
I20260808 07:11:03.421450 16573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:03.421447 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.414040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.18.63.125:45547)
0808 07:11:03.414086 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.414088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.414096 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:03.414115 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.414116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.414124 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.414172 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.414693 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.415130 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.415948 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.416164 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.416237 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.418369 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.418370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.418370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.418377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.420191 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.420576 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.420577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.420577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.420640 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.420843 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.420844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.420844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.421000 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.421175 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.421215 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.421295 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:11:03.421769 16573 log.cc:824] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:03.422839 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: No bootstrap required, opened a new log
I20260808 07:11:03.423043 18685 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 07:11:03.424515 16573 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.424674 16573 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef31b32414f04d48bcc166d474efdc09, State: Initialized, Role: FOLLOWER
I20260808 07:11:03.424053 16318 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "675a64b57a4c4875bb20e1feabf47adb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55580
I20260808 07:11:03.424770 16573 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.425108 16582 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.425148 16573 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:03.425199 16582 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:11:03.427059 16598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:03.427121 16598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:03.427335 16472 master_runner.cc:186] Error getting master registration for 127.18.63.126:44283: OK, instance_id { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" instance_seqno: 1786173063040442 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:11:03.430341 16472 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:11:03.451959 16465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 3
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.125@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:11:03.460438 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.452388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.18.63.124:38155)
0808 07:11:03.452461 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.452463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.452490 (+ 27us) server_negotiation.cc:407] Connection header received
0808 07:11:03.452519 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.452520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.452538 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.452609 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.452941 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.453479 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.454809 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.455010 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.455089 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.457442 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.457443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.457443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.457451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.459084 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.459495 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.459495 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.459496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.459562 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.459773 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.459774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.459774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.459961 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.460082 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.460250 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.460333 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 07:11:03.460539 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.452327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54400)
0808 07:11:03.452388 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.452400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.452456 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.452615 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.452616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.452802 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.452850 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.452852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.453573 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.453575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.454706 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.454710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.455092 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.455093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.455155 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.455683 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.455689 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.457330 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.459173 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.459174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.459174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.459412 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.459632 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.459633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.459633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.459706 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.460084 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.460084 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.460137 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.460215 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.460320 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 07:11:03.462396 16465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: Bootstrap starting.
I20260808 07:11:03.463193 16465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:03.463527 16465 log.cc:824] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:03.464474 16465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: No bootstrap required, opened a new log
I20260808 07:11:03.466099 16465 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.466219 16465 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfb88bfccee8429b9f2f7eb4cce3f5cc, State: Initialized, Role: FOLLOWER
I20260808 07:11:03.466324 16465 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.466632 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.466691 16465 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:03.466735 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: This master's current role is: FOLLOWER
W20260808 07:11:03.468523 16624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:03.468588 16624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:03.468737 16367 master_runner.cc:186] Error getting master registration for 127.18.63.126:44283: OK, instance_id { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" instance_seqno: 1786173063040442 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 07:11:03.471765 16367 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 07:11:03.485579 16359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } attempt: 3
Aug 08 07:11:03 dist-test-slave-bw88 krb5kdc[16245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786173063, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.63.126@KRBTEST.COM for kudu/127.18.63.124@KRBTEST.COM
I20260808 07:11:03.494182 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.485931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.63.124:38155 (local address 127.0.0.1:54404)
0808 07:11:03.485991 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 07:11:03.486003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 07:11:03.486044 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 07:11:03.486215 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 07:11:03.486216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 07:11:03.486419 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 07:11:03.486477 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.486479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.487447 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.487449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.488367 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 07:11:03.488370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.488850 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.488851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 07:11:03.488916 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.489449 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 07:11:03.489455 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 07:11:03.491089 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 07:11:03.493021 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.493022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.493022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.493250 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.493516 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 07:11:03.493517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 07:11:03.493517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 07:11:03.493588 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 07:11:03.493868 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 07:11:03.493869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 07:11:03.493913 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 07:11:03.493979 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 07:11:03.494077 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 07:11:03.494567 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.486010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.18.63.124:38155)
0808 07:11:03.486057 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.486059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.486069 (+ 10us) server_negotiation.cc:407] Connection header received
0808 07:11:03.486103 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.486104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.486115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.486192 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.486560 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.487344 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.488455 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.488766 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.488841 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.491188 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.491190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.491190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.491198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.492932 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.493377 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.493379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.493380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.493443 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.493655 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.493656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.493656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.493786 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.493843 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.494384 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.494467 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 07:11:03.496356 16359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: Bootstrap starting.
I20260808 07:11:03.497162 16359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: Neither blocks nor log segments found. Creating new log.
I20260808 07:11:03.497526 16359 log.cc:824] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: Log is configured to *not* fsync() on all Append() calls
I20260808 07:11:03.498586 16359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: No bootstrap required, opened a new log
I20260808 07:11:03.500008 16359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.500146 16359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675a64b57a4c4875bb20e1feabf47adb, State: Initialized, Role: FOLLOWER
I20260808 07:11:03.500221 16359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.500530 16359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 07:11:03.500599 16318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 07:11:03.500677 16318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:03.501545 16318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.501817 16318 leader_election.cc:288] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [CANDIDATE]: Term 1 election: Requested vote from peers dfb88bfccee8429b9f2f7eb4cce3f5cc (127.18.63.125:45547), ef31b32414f04d48bcc166d474efdc09 (127.18.63.124:38155)
I20260808 07:11:03.502231 16425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "675a64b57a4c4875bb20e1feabf47adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc"
I20260808 07:11:03.502385 16425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:03.503336 16425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 675a64b57a4c4875bb20e1feabf47adb in term 1.
I20260808 07:11:03.503590 16288 leader_election.cc:302] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 675a64b57a4c4875bb20e1feabf47adb, dfb88bfccee8429b9f2f7eb4cce3f5cc; no voters:
I20260808 07:11:03.504117 16531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "675a64b57a4c4875bb20e1feabf47adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ef31b32414f04d48bcc166d474efdc09"
I20260808 07:11:03.504235 16531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 07:11:03.505023 16531 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 675a64b57a4c4875bb20e1feabf47adb in term 1.
I20260808 07:11:03.500532 16627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.505084 16627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:03.505152 16627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 07:11:03.505218 16627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [term 1 LEADER]: Becoming Leader. State: Replica: 675a64b57a4c4875bb20e1feabf47adb, State: Running, Role: LEADER
I20260808 07:11:03.505291 16627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } }
I20260808 07:11:03.505813 16627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675a64b57a4c4875bb20e1feabf47adb. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.505892 16627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:03.506247 16263 master_runner.cc:263] No leader master found from master 675a64b57a4c4875bb20e1feabf47adb
I20260808 07:11:03.506294 16263 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 07:11:03.506527 16641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 07:11:03.506605 16641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 07:11:03.506861 16651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 07:11:03.507035 16651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 07:11:03.508420 16531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [term 1 FOLLOWER]: Refusing update from remote peer 675a64b57a4c4875bb20e1feabf47adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:03.508896 16627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:03.509140 16425 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [term 1 FOLLOWER]: Refusing update from remote peer 675a64b57a4c4875bb20e1feabf47adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 07:11:03.510232 16582 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675a64b57a4c4875bb20e1feabf47adb. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.510334 16582 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:03.510442 16627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 07:11:03.511709 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 675a64b57a4c4875bb20e1feabf47adb. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.511795 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:03.514786 16582 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.514868 16582 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:03.515254 16627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.515391 16627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:03.515508 16655 mvcc.cc:205] Tried to move back new op lower bound from 7316164868128432128 to 7316164868118581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316164868128432128}]
I20260808 07:11:03.515640 16659 mvcc.cc:205] Tried to move back new op lower bound from 7316164868128432128 to 7316164868118581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316164868128432128}]
I20260808 07:11:03.516713 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.516795 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 07:11:03.517117 16627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "675a64b57a4c4875bb20e1feabf47adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675a64b57a4c4875bb20e1feabf47adb" member_type: VOTER last_known_addr { host: "127.18.63.126" port: 44283 } } peers { permanent_uuid: "dfb88bfccee8429b9f2f7eb4cce3f5cc" member_type: VOTER last_known_addr { host: "127.18.63.125" port: 45547 } } peers { permanent_uuid: "ef31b32414f04d48bcc166d474efdc09" member_type: VOTER last_known_addr { host: "127.18.63.124" port: 38155 } } }
I20260808 07:11:03.517194 16627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb [sys.catalog]: This master's current role is: LEADER
I20260808 07:11:03.521804 16651 catalog_manager.cc:1347] Generated new cluster ID: a7dc227367234fd7b958bb4d7a41e973
I20260808 07:11:03.521860 16651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 07:11:03.525298 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.509951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.18.63.125:45547)
0808 07:11:03.509991 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.509993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.517643 (+ 7650us) server_negotiation.cc:407] Connection header received
0808 07:11:03.517742 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.517743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.517753 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.517781 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.518481 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.519577 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.520618 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.520910 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.520950 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.522630 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.522631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.522631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.522639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.524144 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.524499 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.524500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.524500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.524547 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.524742 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.524743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.524743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.524850 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.524932 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.525140 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.525212 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:03.527089 16661 mvcc.cc:205] Tried to move back new op lower bound from 7316164868128432128 to 7316164868118581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316164868128432128}]
I20260808 07:11:03.530241 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.509586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.18.63.124:38155)
0808 07:11:03.509630 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.509632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.515364 (+ 5732us) server_negotiation.cc:407] Connection header received
0808 07:11:03.515475 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.515476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.515488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.515519 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.516360 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.517076 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.518139 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.518431 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.518477 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.526689 (+ 8212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.526690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.526690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.526696 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.528707 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.529302 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.529303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.529303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.529414 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.529626 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.529626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.529627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.529747 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.529826 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.530081 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.530150 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:03.532064 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:03.517703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.18.63.126:44283)
0808 07:11:03.517744 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.517745 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.517999 (+ 254us) server_negotiation.cc:407] Connection header received
0808 07:11:03.518109 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.518110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.518121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.518153 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.518877 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.519588 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.526512 (+ 6924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:03.526807 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:03.526849 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:03.528233 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 07:11:03.528235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 07:11:03.528235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 07:11:03.528241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 07:11:03.529803 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.530715 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.530716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.530716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.530779 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 07:11:03.531368 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 07:11:03.531370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 07:11:03.531370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 07:11:03.531512 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 07:11:03.531597 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:03.531833 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 07:11:03.531911 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 07:11:03.531925 16651 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 07:11:03.532809 16651 catalog_manager.cc:1504] Loading token signing keys...
I20260808 07:11:03.535614 16651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 675a64b57a4c4875bb20e1feabf47adb: Generated new TSK 0
I20260808 07:11:03.535813 16651 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 07:11:03.557289 16580 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.555608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.18.63.124:38155)
0808 07:11:03.555657 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.555659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.556603 (+ 944us) server_negotiation.cc:407] Connection header received
0808 07:11:03.556716 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.556717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.556730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.556761 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.557218 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54426: BlockingRecv error: recv got EOF from 127.0.0.1:54426 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:11:03.558121 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.555991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.18.63.125:45547)
0808 07:11:03.556032 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.556033 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.557583 (+ 1550us) server_negotiation.cc:407] Connection header received
0808 07:11:03.557697 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.557697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.557709 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.557738 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.558047 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41410: BlockingRecv error: recv got EOF from 127.0.0.1:41410 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:11:03.558847 16362 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.556329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.18.63.126:44283)
0808 07:11:03.556376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.556378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.558345 (+ 1967us) server_negotiation.cc:407] Connection header received
0808 07:11:03.558443 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.558444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.558455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.558484 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.558783 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55594: BlockingRecv error: recv got EOF from 127.0.0.1:55594 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 07:11:03.566838 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.565983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.18.63.125:45547)
0808 07:11:03.566023 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.566025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.566258 (+ 233us) server_negotiation.cc:407] Connection header received
0808 07:11:03.566372 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.566373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.566392 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.566422 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.566785 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41416: BlockingRecv error: recv got EOF from 127.0.0.1:41416 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:11:03.573287 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.569696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.18.63.125:45547)
0808 07:11:03.569729 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.569731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.572778 (+ 3047us) server_negotiation.cc:407] Connection header received
0808 07:11:03.572877 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.572878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.572888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.572916 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.573239 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41426: BlockingRecv error: recv got EOF from 127.0.0.1:41426 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:11:03.577178 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.576407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.18.63.125:45547)
0808 07:11:03.576441 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.576443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.576677 (+ 234us) server_negotiation.cc:407] Connection header received
0808 07:11:03.576773 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.576774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.576784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.576812 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.577134 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41440: BlockingRecv error: recv got EOF from 127.0.0.1:41440 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 07:11:03.583115 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.582360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.18.63.125:45547)
0808 07:11:03.582395 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.582397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.582614 (+ 217us) server_negotiation.cc:407] Connection header received
0808 07:11:03.582711 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.582712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.582723 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.582751 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.583057 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41456: BlockingRecv error: recv got EOF from 127.0.0.1:41456 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 07:11:03.593190 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.592358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.18.63.125:45547)
0808 07:11:03.592397 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.592398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.592634 (+ 236us) server_negotiation.cc:407] Connection header received
0808 07:11:03.592730 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.592731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.592742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.592771 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.593138 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41466: BlockingRecv error: recv got EOF from 127.0.0.1:41466 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 07:11:03.611601 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.610631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.18.63.125:45547)
0808 07:11:03.610675 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.610677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.610957 (+ 280us) server_negotiation.cc:407] Connection header received
0808 07:11:03.611068 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.611070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.611095 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.611128 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.611542 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41474: BlockingRecv error: recv got EOF from 127.0.0.1:41474 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:11:03.646090 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.645216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.18.63.125:45547)
0808 07:11:03.645263 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.645265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.645437 (+ 172us) server_negotiation.cc:407] Connection header received
0808 07:11:03.645510 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.645511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.645524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.645601 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.646031 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41478: BlockingRecv error: recv got EOF from 127.0.0.1:41478 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 07:11:03.712020 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.711209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.18.63.125:45547)
0808 07:11:03.711295 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.711297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.711399 (+ 102us) server_negotiation.cc:407] Connection header received
0808 07:11:03.711447 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.711448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.711459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.711525 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.711949 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41494: BlockingRecv error: recv got EOF from 127.0.0.1:41494 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 07:11:03.842157 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:03.841454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.18.63.125:45547)
0808 07:11:03.841530 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:03.841532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:03.841607 (+ 75us) server_negotiation.cc:407] Connection header received
0808 07:11:03.841679 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:03.841680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:03.841692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:03.841739 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:03.842077 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41496: BlockingRecv error: recv got EOF from 127.0.0.1:41496 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 07:11:04.100123 16469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 07:11:04.099408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.18.63.125:45547)
0808 07:11:04.099504 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.099506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.099567 (+ 61us) server_negotiation.cc:407] Connection header received
0808 07:11:04.099628 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.099629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.099643 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.099694 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 07:11:04.100036 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41508: BlockingRecv error: recv got EOF from 127.0.0.1:41508 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 07:11:04.427596 16598 catalog_manager.cc:1259] Loaded cluster ID: a7dc227367234fd7b958bb4d7a41e973
I20260808 07:11:04.427680 16598 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: loading cluster ID for follower catalog manager: success
I20260808 07:11:04.428620 16598 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: acquiring CA information for follower catalog manager: success
I20260808 07:11:04.428939 16598 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef31b32414f04d48bcc166d474efdc09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:11:04.469061 16624 catalog_manager.cc:1259] Loaded cluster ID: a7dc227367234fd7b958bb4d7a41e973
I20260808 07:11:04.469147 16624 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: loading cluster ID for follower catalog manager: success
I20260808 07:11:04.470131 16624 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: acquiring CA information for follower catalog manager: success
I20260808 07:11:04.470480 16624 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dfb88bfccee8429b9f2f7eb4cce3f5cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 07:11:04.616647 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.613612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.18.63.125:45547)
0808 07:11:04.613764 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.613767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.613780 (+ 13us) server_negotiation.cc:407] Connection header received
0808 07:11:04.613853 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.613853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.613931 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.614046 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:04.614237 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.614841 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.615722 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.615955 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.616039 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:04.616231 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:04.616361 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:04.616459 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:04.616509 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 07:11:04.616532 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260808 07:11:04.621093 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.618628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.18.63.124:38155)
0808 07:11:04.618728 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.618739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.618748 (+ 9us) server_negotiation.cc:407] Connection header received
0808 07:11:04.618786 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.618787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.618822 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.618895 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:04.619074 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.619546 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.620356 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.620543 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.620607 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:04.620754 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:04.620853 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:04.620910 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:04.620957 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 07:11:04.620975 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":52,"threads_started":1}
I20260808 07:11:04.625684 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.623069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.18.63.125:45547)
0808 07:11:04.623111 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.623113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.623121 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:04.623146 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.623146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.623156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.623211 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:04.623382 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.623990 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.624923 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.625125 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.625200 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:04.625369 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:04.625405 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:04.625476 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:04.625579 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 07:11:04.625610 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 07:11:04.626314 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.622925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.18.63.124:38155)
0808 07:11:04.622962 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.622964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.623346 (+ 382us) server_negotiation.cc:407] Connection header received
0808 07:11:04.623439 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.623440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.623453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.623510 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:04.623663 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.624351 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.625453 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.625767 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.625838 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:04.626007 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:04.626039 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:04.626082 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:04.626221 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 07:11:04.626243 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 07:11:04.626566 16780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 07:11:04.622953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.18.63.126:44283)
0808 07:11:04.623148 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 07:11:04.623150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 07:11:04.623158 (+ 8us) server_negotiation.cc:407] Connection header received
0808 07:11:04.623225 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 07:11:04.623226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 07:11:04.623277 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 07:11:04.623354 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 07:11:04.623515 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.624306 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.625472 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 07:11:04.625884 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 07:11:04.625953 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 07:11:04.626185 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 07:11:04.626277 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 07:11:04.626329 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 07:11:04.626427 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 07:11:04.626458 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":60,"threads_started":1}
I20260808 07:11:04.628194 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 16263
I20260808 07:11:04.639166 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 16367
I20260808 07:11:04.648643 18685 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskERtyUc/build/release/bin/kudu with pid 16472
2026-08-08T07:11:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3066 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6182 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (285347 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST